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ABSTRACT 

This  study  investigated  the  formal  teacher  evaluation 
practices  of  superintendents,  other  central  office  staff, 
and  princ ipals  . 

A  questionnaire  collected  data  on  their  evaluation 
practices  between  September  1983  and  April  1984:  (1) 
demographic  data  on  the  responding  evaluator;   (2)  data 
related  to  initiating  factors,   purposes,  and  outcomes  of 
their  last  teacher  evaluation;  and   (3)  data  on  the 
evaluative  procedures  they  used  in  that  evaluation. 

Experts  in  "due  process"  and  "improvement  of 
instruction"  adjudicated  items  in  part  3  to  establish 
minimum  criteria  for  satisfactory  evaluation. 

Respondents  generally  placed  a  high  priority  on  teacher 
evaluation,   but  had  inadequate  time. 

Less  than  half  of  the  respondents  followed  board  policy 
"strictly  to  the  letter."     Two-thirds  of  the  respondents 
gave  a  positive  rating  to  the  last  teacher  evaluated. 

Of  the  teachers  evaluated,   60.8%  were  tenured,  and 
39.2%  were  first  year  teachers,  or  new  to  the  school/system. 
Junior  high  teachers  comprised  the  largest  group  evaluated, 
and  language  arts  and  math  were  the  subjects  most  observed. 

The  majority  of  evaluations  were  initiated  as  routine 
evaluations,  with  primary  purposes  to:     improve  the 
teacher's  performance,   award  a  permanent  contract,  provide  a 
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basis   for  retention   in  position,   or  recommend  permanent 
certification.     As  a  result,  most  teachers  were  retained  in 
their  present  position,   received  a  permanent  contract,  or 
were  recommended  for  permanent  certification. 

Very  few  of  the  respondents  (4.5%)  met  the  minimum 
criteria  to  assure  due  process  and  fewer  (1.3%)  met  the 
criteria  for  improvement  of  instruction. 

The  completeness  of  the  evaluation   (i.e.,   score  on  part 
3  of  the  questionnaire)  correlated  positively  with  time 
spent  in  classroom  observation,   number  of  observation 
visits,  priority  of  teacher  evaluation,   and  number  of 
related  university  courses. 

Where  the  purpose  or  outcome  of  the  evaluation  related 
to  inservice  needs,   the  completeness  of  the  evaluation 
correlated  positively  with  the  purpose  or  outcome. 

Where  the  purpose  or  outcome  of  the  evaluation  related 
to  a  recommendation  for  permanent  certification,  the 
completeness  of  the  evaluation  correlated  negatively  with 
the  purpose  or  outcome . 

No  relationship  was  found  between  the  presence  of  a 
written  policy  on  formal  teacher  evaluation  and  the 
completeness  of  the  evaluation,   nor  was  the  level  of 
adherence  to  policy  significantly  related  to  completeness. 
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CHAPTER  I 
INTRODUCTION 

Evaluation  in  education  is  universally  endorsed,  widely 
misunderstood,  poorly  planned,  crudely  implemented, 
haphazardly  staffed,   and  even  ignored  when  the  results 
are  in!     Nonetheless,   supervisors  are  coming  to  accept 
responsibility  for  the  evaluation  task  by  recognizing 
that  it  is  essential  for  guiding  the  improvement  of 
instruction  process.      (Harris,    1985,   p.  189) 

In  December  1982,  James  Keegstra,  a  social  studies 
teacher  at  the  Eckville  High  School,   Eckville,   Alberta  was 
dismissed  from  his  position  as  a  teacher  in  that  school.  At 
a  subsequent  Board  of  Reference  hearing  held  in  the  Judicial 
District  of  Edmonton  in  April  1983,   it  came  to  light  that 
Keegstra  had,   for  a  protracted  period,  been  teaching  a 
social  studies  program  which  differed  substantially  from  the 
authorized  Alberta  curriculum  and  which  was  perceived  to  be 
anti-Jewish  in  content.     It  was  adequately  demonstrated  that 
Keegstra 's  students  had  accepted,  as  fact,   the  distorted  and 
biased  view  of  history  which  had  been  presented  to  them 
(Keegstra  v.  County  of  Lacombe ,   1983,  pp.  19-21). 

The  Keegstra  case  naturally  attracted  a  great  deal  of 
attention  in  the  educational  community  and  among  the  general 
public.     Subsequently,   the  Honourable  David  King,  Minister 
of  Education,  established  a  Minister's  Consultative 
Committee  on  Tolerance  and  Understanding. 


2 


Ron  Ghitter,   the  Chairman  of  the  Minister's  Committee, 

in  addressing  delegates  at  the  Alberta  School  Trustees' 

Association  Convention  on  November  29,   1983   in  Calgary, 

Alberta  stated: 

As  unforgivable  as  the  actions  of  the  Eckville  high 
school  teacher   (Jim  Keegstra)  may  be,   Eckville  has 
raised  our  level  of  consciousness.     In  my  view, 
Eckville  illustrated  an  institutionalization  of  our 
educational  system  which  has  made  it  unresponsive  and 
keeps  it  from  acting  in  a  responsive  manner. 

In  my  mind,   Eckville  raises  the  question  again  of 
teacher  evaluation.    .    .    .  and  the  procedures  utilized 
in  this  province  with  respect  to  discipline  of 
teachers,   termination  of  teachers,  and  the  whole 
process  of  teacher  standards,   the  encouragement  and 
rewarding  of  teachers  with  merit,  and  the  ridding  from 
our  system  of  teachers  who  are   incompetent.    .    .  . 

I  am  aware  of  this  association's  concerns  with  respect 
to  teacher  evaluation.     In  many  areas  of  this  province, 
we  have  seen  school  boards  who  can  proudly  boast  of 
evaluation  programs  within  their  systems  that  are 
thoughtful,   that  are  fair,   and  are  designed  to  ensure 
that  they  are  acceptable,   and   I  would  speculate  even 
welcomed  by  the  teachers  within  the  system. 

Within  other  school  boards   in  this  province,  we  have 
experienced  little  in  the  way  of  effective  evaluation. 
Within  many  jurisdictions,   the  teachers  could  go  for 
seven  to  10  years  without  ever  being  evaluated,   if  at 
all.    .    .  . 

Red  herrings  are  dragged  across  the  issue,  with  such 
discussion  as  who  will  do  the  evaluation,   how  can  it  be 
effective,  why  do  we  need  evaluation  when  we  can  rely 
on  the  integrity  of  the  individual  teacher  and  on  and 
on  .... 

But  the  fact  remains  that  if  standards  are  to  be 
achieved  and  if  teachers  are  to  be  recognized  for  the 
excellent  work  they  do  in  our  schools  there  must  be  a 
place  for  evaluation.     (pp.  20-24) 

Since  1970,   the  prime  responsibility  for  teacher 

evaluation  has  rested  with  the  local  jurisdiction,   and  an 
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Alberta  Education  Evaluation  Policy   issued  in  May  1984 
reiterated  this  position   (Alberta  Education,   1984,   p.  66). 

Studies  conducted  in  1977   (Holdaway,   1977;  Reikie, 
1977)   found  that,   although  some  83%  of  the  114  Alberta 
school  jurisdictions  surveyed  evaluated  teachers  new  to  the 
system,   tenured  teachers  were  evaluated  far  less  frequently. 
In  fact,  only  53%  of  jurisdictions  evaluated  tenured 
teachers  on  any  kind  of  a  scheduled  basis   (e.g.,  once  every 
three  years).     A  survey  conducted  by  the  Planning  and 
Research  Branch  of  Alberta  Education   in  September  1980 
showed  that,  at  that  time,   90%  of  school  jurisdictions 
evaluated  first  year  teachers  but  that  33%  of  jurisdictions 
had  no  scheduled  basis  for  the  evaluation  of  tenured 
teaching  staff.     The  same  study  also  indicated  that,  of 
those  jurisdictions  indicating  that  they  had  a  written 
evaluation  policy,  only  55%  reported  that  they  carried  out 
the  pol icy . 

The  implications  of  these  studies  are  far  reaching. 
They  suggest  that,   although  policies  may  require  teachers  to 
be  evaluated  according  to  some  fixed  schedule,   in  practice 
this  may  not  occur  and  thus  it  is  entirely  possible  that  a 
new  teacher  may  be  granted  tenure  without  a  formal 
evaluation  of  performance,   and  a  tenured  teacher  may  teach 
for  many  years  without  ever  being  formally  evaluated. 

Is  formal  evaluation  necessary ?     Isn't  a  trained 
teacher  in  possession  of  an  appropriate  teaching  credential 
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in  the  best  position  to  judge  his/her  own  performance? 
Isn't  an  external  evaluation  of  a  teacher  a  frightening, 
inhumane,   and  unnecessary  experience  for  a  highly  trained 
professional? 

The  Keegstra  case  amply  demonstrates  what  can  happen 

when  teachers  are  insufficiently  monitored  and  Millman 

(1981)  aptly  summarizes  the  point: 

Teaching  is  not  a  solitary  activity  affecting  no  one. 
On  the  contrary,   the  lives  of  many  students  are  altered 
in  far-reaching  and  significant  ways  by  the  instructors 
with  whom  they  interact.     Teaching   is  too  important  to 
too  many  to  be  conducted  without  a  critical  inquiry 
into  its  worth.     Besides,  court  cases  have  made  it 
clear  that  students  have  rights,   and  schools  and 
colleges  have  a  responsibility  to  ensure  the  quality  of 
their  curriculum  and  instruction.     The  protection  of 
rights  and  the  fulfillment  of  responsibility  require 
that  teacher  self-assessment  be  supplemented  by 
evaluation  by  others.     (p.  12) 

Alberta  Education  states  that  "Evaluation  of  teacher 

performance   is  viewed  as  a  vital  component  in  ensuring  that 

quality  educational  programs  are  provided  to  Alberta 

children"   (1982,  p.  8)  and  proposes  eight  purposes  for 

teacher  evaluation: 

1.     To  promote,   achieve,  and  maintain  an  acceptable 
quality  of  instruction. 


2.  To  help  improve  performance  of  teachers   in  securing 
desirable  instructional  outcomes. 

3.  To  provide   information  which  will  be  useful  when 
considering  placement  of  staff,  transfer, 
retention,  promotion,   tenure,   and  permanent 
cert i  f  icat ion . 

4.  To  provide  specific   feedback  concerning  teacher 
performance . 
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5.  To  provide  professional  assistance  to  teachers  in 
the  performance  of  teaching  tasks. 

6.  To  provide  a  basis  for  planning  in-service 
programs  . 

7.  To  provide  written  evaluation  reports  on  teachers 
in  public  and  private  schools  for  purposes  of 
documentation,  as  required. 

8.  To  assist  teachers  in  professional  growth  and 
development.     (p.  8) 

Of  these  purposes,    items  1,   2,   4,   5,   6,   and  8 
essentially  relate  to  the  quality  or  improvement  of 
instruction  and  items  3  and  7  refer  to  the  documentation 
functions  of  evaluation. 

The  Holdaway   (1977)  and  Reikie   (1977)   studies  showed 
that  in  1977,  school  jurisdictions  used  evaluation  primarily 
for  documentation:     the  awarding  of  permanent  contracts 
(94%),   recommendation  for  permanent  certification   (96%),  and 
providing  a  basis  for  dismissal   (83%).     Evaluation  was  used 
to  assist  in  improving  teacher  performance  in  only  18%  of 
school  jurisdictions. 

The  Alberta  Education  study  of  1980  showed  a  marked 
change  in  the  focus  of  teacher  evaluation  in  a  number  of 
areas.     Certification  and  the  awarding  of  a  permanent 
contract  remained  primary  uses   (90%),   but  use   for  the 
improvement  of   instruction  had  risen  from  18%   in  1977  to  83% 
in  1980. 

Thus,  we  see  the  two  major  purposes  for  evaluation  in 
Alberta,    (1)   the  improvement  of  instruction,  commonly  called 
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formative  evaluation  and   (2)  documentation  for  the  purpose 
of  staff  placement,   retention,   transfer,  promotion,  tenure, 
and  permanent  certification,   commonly  called  summative 
evaluation.     In  the  former  case,   the  evaluation  strategies 
should  be  congruent  with  those  that  lead  to  improved 
performance,   and  in  the  latter  case  the  evaluation  practices 
must  be  fair  and  reasonable  and  allow  the  teacher  "due 
process  . " 

This  situation  is  reflected  in  Alberta  Education's 
Teacher  Evaluation  Policy  which  states  that  teachers  will  be 
evaluated  to  "assist  in  the  provision  of  effective 
instruction  to  students  and  in  the  professional  growth  and 
development  of  teachers"   (Alberta  Education,   1984,  p.  66). 
The  policy  requires  that  the  results  of  the  evaluation  be 
used: 

1.  to  assist  in  the  professional  development  of 
teachers  and, 

2.  to  take  appropriate  action  with  respect  to  teachers 
whose  performance  is  unacceptable. 

In  addition,   the  policies  developed  by  School  Boards  should 

be  fair  and  consistent  and  provide  for  due  process. 

A  comparison  of  the  results  of  the  Holdaway   (1977)  and 

Reikie   (1977)   studies  with  those  of  the  Alberta  Education 

study  indicated  a  distinct  change   in  the  emphasis   in  teacher 

evaluation.     However,   both  studies  sought  to  establish  the 

current  situation  in  Alberta  by  questioning  School 

Superintendents  on  the  policies  and  practices  of  their 
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jurisdiction  in  preparing  formal  evaluations  of  teachers. 

These  studies  were  limited   in  that  they  recorded  only  the 

superintendent's  perceptions  of  the  situation  and  Reikie 

(1977)  regards  this  as  a  "significant  limitation"  to  his 

study  and  suggests: 

Future  studies  might  examine  teacher  evaluation 
policies  and  practices  from  the  perspective  of  other 
personnel   involved  in  the  evaluation  process  such  as 
school  principals,   teachers  and  other  central  office 
personnel.      (p.  95) 

He  also  suggests  that: 

More  specific  descriptive   information  might  initiate 
the  development  of  viable  proposals  for  change  and 
improvement.     (p.  56) 

Clearly  a  situation  exists   in  Alberta   in  which  the 
actual  "on  site"  evaluation  practices  are  largely 
undocumented,   and  the  current  available  data  are  limited  to 
superintendents'  perceptions.     The  situation  has  led  to  a 
renewed  public  and  political   interest  in,   and  pressure  for, 
increased  monitoring  of  teachers. 

The  Alberta  School  Trustees'  Association  and  the 
Alberta  Teachers'   Association  have  recent,  or  recently 
revised,  position  papers  on  teacher  evaluation  or 
performance  appraisal;  and  Alberta  Education  has  recently 
(May  1984)   introduced  a  set  of  evaluation  policies  which 
include  teacher  evaluation. 

It  is  abundantly  clear  that  all  concerned  parties  in 
Alberta  are  ready  for  a  change,   but  a  change  from  what? 
Before  any  meaningful  change  in  teacher  evaluation  practices 
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can  be   initiated,   a   thorough  understanding  of  the  status  quo 
is  necessary.     As   Peikie   (1977)   said,    "A  great  need  exists 
for  a  proportionately  larger  volume  of  research  of  an 
empirically  descriptive  nature"   (p.  2). 

The  Purpose  of  the  Study 

The  primary  purpose  of  this  study  was  to  describe  the 
state  of  formal  teacher  evaluation  in  Alberta  in  the  1983- 
1984  school  year.     The  specific  aims  were: 

1.  to  collect  demographic   information  about  the 
evaluators  and  teachers   involved   in  formal  teacher 
evaluation  in  Alberta. 

2.  To  describe  the  initiating  factors,  intended 
purposes,  and  outcomes  of  formal  teacher  evaluation. 

3.  To  describe  the  procedures  used  by  evaluators  in 
formally  evaluating  teachers. 

4.  To  analyze  the  completeness  of  the  teacher 
evaluation  procedures  used  by  evaluators  in  Alberta  with 
respect  to: 

(a)  the   improvement  of  instruction. 

(b)  due  process  requirements. 

5.  To  identify  the  relationships  between  demographic 
factors  and  the  completeness  of  evaluation. 

Justification  for  the  Study 

The  Keegstra  Case  has,  as  Ghitter  stated,    "raised  our 
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level  of  consciousness"    (1983,   p.   20)  with  respect  to  what 
happens   in  the  classroom.     The  case  demonstrated,   all  too 
clearly,   the   inadequacy  of  some  of  the  teacher  evaluation 
practices   in  Alberta. 

Judicious  practice  would  suggest  that  School  Boards  in 
Alberta  should  have  written  policy  on  teacher  evaluation, 
and  Alberta  Education,   in  May  1984,  published  policies  which 
will  require  that  each  school  jurisdiction   "develop  and 
adopt  written  policies,  guidelines  and  procedures"   (p.  66) 
for  the  evaluation  of  teachers. 

This  study  is  justified  on  the  basis  that  it  will 
provide  information  to  Alberta  Education,   School  Boards, 
administrators,   teachers,  professional  organizations,  and 
the  public  at  large.     That  information  is  important  because 
it  can  provide  a  basis  for  decision  making   in  the  areas  of 
policy  making,   in  service  training,   and  in  the 
implementation  of  more  effective  teacher  evaluation 
practices . 

Assumptions 

The  following  assumptions  were  made: 

1.  Responses  to  the  study  were  made  in  good  faith. 

2.  The  questions  on  the  questionnaire  were  not 
interpreted  ambiguously. 

3.  The  data  received  on  the  questionnaires  accurately 
reflected  the  beliefs  and  practices  of  the  respondents. 
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4.     The   last  formal  teacher  evaluation  carried  out  by 
each  of  the  respondents  was  typical  of  their  usual 
evaluative  practice. 

Limitations 

1.  The  study  reflected  the  evaluative  practices  only 
as  they  applied  to  the  formal  evaluation  of  teachers  in 
Alberta  during  the  period  September  1,   1983  to  April  30, 
1984  . 

2.  The  study  reflected  the  evaluative  practices  only 
to  the  extent  that  "the  last  formal  teacher  evaluation 
carried  out"  exemplified  the  normal  evaluative  practice  of 
each  respondent. 

Definition  of  Terms 

Alberta  Education.     Alberta  Education  is  that  branch  of 
the  Alberta  Provincial  Government  responsible  for  the 
provision  of  educational  services  at  the  primary, 
elementary,   and  secondary  levels. 

Board .     Board  means  a  board  of  trustees  of  a  district 
or  division. 

Board  of  Reference.     Board  of  Reference  means  the 
appeal  body  established  pursuant  to  sections  95  to  99  of  The 
School  Act. 

Conference  of  Alberta  School  Superintendents. 
Conference  of  Alberta  School  Superintendents  means  that 
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organization  whose  meinber sh ip  consists  of  superintendents 
and  other  central  office  staff  who  are  not  full  members  of 
the  Alberta  Teachers  Association. 

County .     County  means  those  school  districts  under  the 
jurisdiction  of  a  County  Council  pursuant  to  The  County  Act. 

County  Act.     County  Act  means  Chapter  C-27  of  the 
Revised  Statutes  of  Alberta  1980. 

District .     District  means  a  school  district  established 
pursuant  to  The  School  Act. 

Divis ion .     Division  means  a  school  division  (consisting 
of  any  number  of  school  districts)  established  pursuant  to 
The  School  Act. 

Formal   Evaluation.     Formal  evaluation  means  an 
evaluation  of  a  teacher  that  includes  a  written  report 
leading  to  a  recommendation  or  rating. 

Other  Central  Office  Staff.     Other  central  office  staff 
means  those  people  employed  by  a  school  jurisdiction  who 
were  members  of  the  Conference  of  Alberta  School 
Superintendents  as  of  December  1983. 

Public  School  District.     Public  School  District  means  a 
school  district  established  pursuant  to  The  School  Act. 

School  Act.     School  Act  means  Chapter  S-3  of  the 
Revised  States  of  Alberta  1980. 

School  Jurisdiction.     School  jurisdiction  means  a 
public  school  district  or  division,   a  separate  school 
district,  or  a  County. 
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Separate  School  District.     Separate  school  district 
means  a  school  district  established  pursuant  to  The  School 
Act.     Separate  school  districts  may  be   formed  within  the 
boundaries  of,  or  co-terminus  with  public  school 
jurisdictions.     If  the  public  school  jurisdiction  is  a  non- 
Catholic  jurisdiction,   the  separate  district  will  provide  a 
Roman  Catholic  education- — and  vice  versa.     The  majority  of 
separate  school  districts  in  Alberta  are  Roman  Catholic. 

Superintendent .     Superintendent  means  the 
Superintendent  of  Schools  of  a  school  jurisdiction  appointed 
by  a  Board  pursuant  to  The  School  Act. 

Teacher  means  a  person  who  holds  a  certificate  of 
qualification  issued  under  the  Department  of  Education  Act 
and  is  employed  by  a  school  jurisdiction  to  instruct 
students  in  a  school. 

Tenured  Teacher.     Tenured  teacher  means  a  teacher  who 
has  been  granted  a  continuous  contract  pursuant  to  The 
School  Act. 
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CHAPTER  II 

REVIEW  OF   SELECTED  LITERATURE 

The  critics  of  the  educational  establishment  are  far 
from  ready  to  see  the  abandonment  of  overt  appraisals 
of  teaching  performance.     There  remains  a  strong 
feeling  that  formal  evaluation  is  an  essential 
guarantee  that  standards  will  be  maintained.     In  actual 
fact,   however,   such  a  belief  is  largely  illusory. 
(Fleming,   1971,   p.  445) 

The  central  purpose  of  this  study  is  to  describe  the 
state  of  teacher  evaluation  in  Alberta,   and  to  this  end, 
this  review  of  the  literature  will  focus  upon  the  research 
related  to  supervisory  and  evaluative  processes. 

Evaluation  and  Supervision  Defined 

The  literature  contains  a  plethora  of  definitions  of 
supervision  and  evaluation  and  the  line  dividing  the  meaning 
of  these  two  terms  is  unclear. 

Some  of  the  very  early  definitions  of  supervision  such 
as  those  cited  by  Earr,   Burton  and  Brueckner   (1947)  were 
meaningless  and  today  seem  quite  humorous:     "the  business  of 
the  supervisor  is  to  cast  a  genial  influence  over  his 
schools,   but  otherwise  he  is  not  to  interfere  with  the  work" 
or  "supervision  is  the  vision  in  the  old  and  beautiful  sense 
of  seeing  things  invisible." 

Many  of  the  more  recent  definitions  of  supervision 
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relate   to  its   function   in  the   improvement  of  instruction 
(Acheson  &  Gall,   1980;   Alberta   Education,   1984;  Alberta 
Teachers'   Association,   1980;    Burton,    1922;    Burton  Sf 
Brueckner,   1955;   Harris,   1975;   Spears,   1953)   yet  many  of  the 
same  authors  acknowledge  the  need  for  supervisors  to  make 
judgements  about  the  performance  of  a  teacher  and  include 
this  V7ithin  their  definition.     For  instance.   Burton  (1922) 
includes  the  rating  of  teachers  in  his  definition  and 
Acheson  and  Gall   (1980)   include  the  evaluation  of  teachers 
for  promotion,   tenure,  or  other  decisions  as  one  of  their 
goals  for  clinical  supervision. 

Since  the  terms  supervision  and  evaluation  are  often 
used  synonymously  in  the  literature,   this  review  will 
examine  the  writing  from  that  perspective. 

The  Evolution  of  Evaluation 

Historically  the  philosophy  underlying  evaluation  has 
undergone  a  number  of  significant  changes.     Burton  and 
Brueckner   (1955)  describe  the  selection  of  teachers  in  the 
Massachusetts  Bay  Colony  in  1654  wherein  town  selectmen 
secured  teachers  of  "certain  religious  and  moral  qualities." 
By  the  early  1700s,   inspection  had  appeared  and,   in  Boston, 
committees  of  citizens  were  appointed  to  visit  and  inspect 
the  schools. 

Towns  grew  and  schools   increased  in  size  until  several 
teachers  were  working  in  one  building.     This  resulted   in  the 
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establishment  of  the  "principal  teacher"  who  performed 
certain  administrative  and  managerial   functions.  However, 
the  supervisory  functions  were  not  delegated  to  principals 
until  much  more  recently.     By  the  middle  of  the  nineteenth 
century  the  office  of  "superintendent  of  schools"  had 
appeared  and,  while   in  some  cases  he  assumed  supervisory 
responsibilities,   school  boards  continued  to  retain  most  of 
the  supervisory  or  inspectoral  power. 

Burton  and  Brueckner   (1955),   Lucio  and  McNeil  (1962), 
and  Wiles  and  Lovell   (1967)   identify  a  number  of  stages  in 
the  evolution  of  supervision  in  the  twentieth  century.  The 
turn  of  the  century  saw  supervision  by  specialists  as  new 
subjects  were  added  to  the  curriculum.     These  supervisors 
were  travelling  teachers  rather  than  lay  people.  Scientific 
supervision  was  the  thrust  of  the  1920s.     "Research  and 
measurement  were  to  be  the  domain  of  supervision.  Teachers 
were  to  apply  findings"   (Lucio  &  McNeil,  1962). 

During  the  1930s  and  40s,   the  emphasis  was  upon 
democratic  supervision  wherein  the  supervisor  appealed  to 
the  feelings  and  emotions  of  the  teacher  to  achieve  the 
desired  goals.     In  the  late  1940s,   this  developed  into  a 
cooperative  model   in  which  the  supervisory  process  was  seen 
as:      (1)   the  facilitation  of  interaction  and  mutual 
assistance  among  staff   (Wiles  &  Lovel,   1967)   and   (2)  the 
encouragement  of  wider  participation  of  all  the  stakeholders 
in  the  educational  endeavor  in  the  formulation  of  mutually 
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accepted  educational  goals   (Lucio  &  McNeil,  1962). 

Lucio  and  McNeil   (1962)   describe  the  1950s  as  the  age 
of  supervision  through  reason  and   intelligence  which  they 
describe  as   "The  specification  of  desired  outcomes  and 
appropriate  behavior  necessary  to  their  attainment   .    .  . 
Action  through  wide  participation  of  all  concerned  in  the 
processes  of  inquiry  and  the  judgement  of  outcomes"   (p.  12). 

During  the  1960s,   Amidon  and  Flanders   (1963),  Flanders 
(1964),  and  Amidon  and  Hunter   (1966)  were  leaders  in  the 
development  of  strategies  designed  to  gather  observational 
data  so  as  to  provide  teachers  with  meaningful  information 
for  the  purpose  of  changing  or  reinforcing  behavior. 
However,   the  most  significant  development  in  the  supervision 
of  teachers  in  the  last  twenty  years  is,   according  to 
Sullivan  (1980),   the  growth  of  the  clinical  supervision 
model  developed  by  Morris  Cogan  and  Robert  Goldhammer,  a 
process  which  Reavis   (1977)   sees  as  combining  the 
fundamental  precepts  of  the  humanistic  approach,  democratic 
practices  and  the  scientific  method. 

Thus  we  have  seen  a  dramatic  change  in  the  focus  of 
evaluation,   a  change  from  the  traditional   inspectoral  model 
to  a  more  democratic,   analytical  model.     Burton  and 
Brueckner   (1955,   p.   13)  use  "prominent  catch  words"  to 
differentiate  between  traditional  and  modern  styles  of 
supervision: 


17 


Traditional 


Modern 


1  .     Inspection  . 

2.  Teacher-f ocussed . 

3.  Visitation  and  conference 


4.     Random  and  haphazard,  or 
a  meager,   formal  plan. 


5.  Imposed  and  authoritarian 

6.  One  person  usually. 
Thus : 


1  .     Study  and  analysis 

2.  Focussed  on  aim, 
method,  pupil,  and 
environment . 

3.  Many  diverse 
functions . 

4 .  Definitely 
organized  and 
planned  . 

5.  Derived  and 
cooperative . 

6  .     Many  persons . 


Traditional  supervision  consists  largely  of  inspection 
of  the  teacher  by  means  of  visitation  and  conference, 
carried  on  in  a  random  manner,  with  directions  imposed 
on  the  teacher  by  authority  and  usually  by  one  person. 
Modern  supervision,   by  contrast  involves  the  systematic 
study  and  analysis  of  the  entire  teaching-learning 
situation  utilizing  a  carefully  planned  program  that 
has  been  cooperatively  derived  from  the  situation  and 
which  is  adapted  to  the  needs  of  those  involved  in  it. 
(Burton  &   Brueckner,   1955,   p.  13) 


Purposes  of  Evaluation 


The  purposes  of  evaluation  have  been  discussed  by  many 
researchers;   as  early  as  1922  Burton   (1922,   pp.   9,  10) 
suggested  that  it  dealt  with  the  improvement  of  the  teaching 
art  and  the  improvement  of  teachers  in  service.     Burton  and 
Brueckner   (1955)   identify  the  purposes  of  supervision  as  "an 
expert  technical  service  primarily  aimed  at  studying  and 
improving  cooperatively  all  factors  which  affect  child 
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growth  and  development"    (p.   10)   and  Harris   (1985)   says  that 
supervision  "is  what  school  personnel  do  with  adults  and 
things  to  maintain  or  change  the  school  operation  in  ways 
that  directly  influence  the  teaching  processes  employed  to 
promote  pupil  learning"   (p.  10). 

The  Alberta  School  Trustees  Association   (1982)  states 
"The  major  objective  of  the  program  (or  performance 
appraisal)   is  desired  and  specified  improvement  in  teaching 
performance"  and  Wiles  and  Lovell   (1967)  would  also  include 
the  development  of  human  relations   in  his  purposes: 
"supervision  consists  of  all  the  activities  related  to 
morale,   improving  human  relations,   inservice  education,  and 
curriculum  development"   (p.  5). 

Acheson  and  Gall   (1980)   identify  the  major  aim  as  the 
improvement  of  teachers'  classroom  instruction  and,  in 
addition,   list  the  following  specific  goals  of  clinical 
supervision : 

1.  To  provide  teachers  with  feedback  on  the  current 
state  of  their  instruction. 

2.  To  diagnose  and  solve   instructional  problems. 

3.  To  help  teachers  develop  skill  in  using 
instructional  strategies. 

4.  To  evaluate  teachers  for  promotion,   tenure  and 
other  decisions. 

5.  To  help  teachers  develop  a  positive  attitude  about 
continuous  professional  development.     (pp.  12-14) 

Similarly,  other  authors  have   identified  a  variety  of 

purposes.     Alberta  Education   (1982)    identified  eight 


19 


purposes  somewhat  similar  to  those  enumerated  by  Acheson  and 
Gall   (1980);   Patterson   (1974)   and  the  Alberta  Teachers' 
Association   (1980a)    identify  two  primary  purposes: 

1.  For  the   improvement  of  the  teacher's  instructional 
performance  and, 

2.  For  judgemental  functions  related  to  decision 
making  and  teacher  competency. 

These  two  primary  purposes  may  be  categorized  as 

formative  and  summative   (Wittrock  &  Wiley,   1970;   Cooley  & 

Lohnes,   1976).     Formative  evaluations  are  performed  with  the 

intention  of  assisting  teachers  to  improve  their 

performance;   summative  evaluations  culminate  in  reports 

which  are  used  as  a  basis  for  judgementally  rating  the 

teachers. 

Darling-Hammond,  Wise,   and  Pease   (1983),   in  an 
extensive  review  of  the  literature,   identify  four  basic 
purposes  for  teacher  evaluation:     individual  staff 
development,   individual  personnel   (job  status)  decisions, 
school   improvement,   and  school  status   (e.g.,  certification) 
decisions.     They  suggest  that  "many  teacher  evaluation 
systems  are  nominally  intended  to  accomplish  all  four  of 
these  purposes,   but  different  processes  and  methods  are 
better  suited  to  one  or  another  of  these  objectives"  (p. 
302  ). 

In  general,   teacher  evaluation  processes  most  suited  to 
accountability  purposes  must  be  capable  of  yielding 
fairly  objective,   standardized,  and  externally 
defensible  information  about  teacher  performance. 
Evaluation  processes  useful  for  improvement  objectives 
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must  yield  rich,  descriptive  information  that 
illuminates  sources  of  difficulty  as  well  as  viable 
courses  for  change.     Teacher  evaluation  methods 
designed  to  inform  organizational  decisions  must  be 
hierarchically  administered  and  controlled  to  ensure 
credibility  and  uniformity.     Evaluation  methods 
designed  to  assist  decision  making  about  individuals 
must  consider  the  context  in  which  the  individual 
performance  occurs  to  ensure  appropriateness  and 
sufficiency  of  data.     (Darling-Hammond  et  al  .  ,  1983, 
p.  303) 

Evaluation  Processes 

In  discussing  teacher  evaluation,   the  Alberta  School 
Trustees  Association   (1982)  differentiates  between  teacher 
competence  and  teacher  performance.     "Competence,  as  defined 
by  Webster,   is    'having  requisite  or  adequate  ability  or 
qualities;   legally  qualified  or  adequate.'     Performance  on 
the  other  hand,   is  defined  as    'execution  of  an  action; 
something  accomplished'"   (p.  1).     The  Association  postulates 
that  it  is  the  performance  area  that  should  be  the  subject 
of  teacher  evaluation. 

Medley   (1982)   suggests  that  teacher  attributes  may  be 
categorized  into  teacher  competency,   teacher  competence, 
teacher  performance,   and  teacher  effectiveness.  Competency 
refers  to  any  skill  or  piece  of  knowledge  possessed  by  the 
teacher  and  relevant  to  the  practice  of  teaching,  competence 
relates  to  the  competencies  a  teacher  possesses,  performance 
deals  with  what  the  teacher  does  on  the  job,   and  is 
situational  in  context,  and  effectiveness  refers  to  the 
effect  a  teacher's  performance  has  on  students. 
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The  processes  used  by  evaluators  to  assess  teacher 
competence,  performance,  and  effectiveness  are  varied. 
Levin   (1979)   identified  six  methods  of  evaluating  teachers. 
A  similar,  more  comprehensive  list  of  eight  processes  has 
been  compiled  by  Darling-Hammond  et  al .    (1984,   pp.  304-308) 
and  the  main  headings  are  presented  here  together  with  a 
condensed  and  paraphrased  version  of  their  descriptions. 

1 .  Teacher  Interviews.     Teacher  interviews  or 
conferences  have  been  widely  used  in  the  past  and 
are  an  essential  component  of  some  of  the  modern 
evaluation  strategies.     In  the  past  the  interviews 
were  often  used  for  communicating  performance 
appraisals  or  in  the  hiring  or  firing  of  teachers. 
Today,   the  involvement  of  teachers  in 
preobserva t ion  conferences  as  well  as 

pos tobservation  interviews  is  seen  as  a  useful  tool 
in  promoting  professional  growth. 

2.  Competency  Tests.     There  is  a  growing  trend  for  the 
use  of  standardized  tests  in  the  initial 
certification  and  hiring  of  teachers  and  an 
increasing  belief  that  such  tests  have  value  for 
recerti f ication  and  dismissal  decisions.     The  most 
widely  used  test  is  the  National  Teacher 
Examination  which  is  administered  to  as  many  as 
75,000  teacher  candidates  in  24  states  and  311 
school  districts  each  year. 

3.  Indirect  Measures.     Indirect  measures  include 
training,  experience,   commitment  to  extracurricular 
activities  etc.  and  while  such  measures  are  linked 
to  salary  and  promotion,   there  is  no  evidence  to 
suggest  that  they  differentiate  between  effective 
or  ineffective  teachers. 

4.  Classroom  Observations.     Coupled  with  teacher 
interviews  or  conferences,   this  method  is 
undoubtedly  the  mainstay  of  current  teacher 
evaluation  practice.     Holley  and  Evertson   (1981,  p. 
90)   say  that  classroom  observation  reveals   "a  view 
of  the  climate,   rapport,   interaction  and 
functioning  of  a  classroom  available  from  no  other 
source . " 
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5.  Student  Ratings.     Student  ratings  are  commonly  used 
at  the  college  level  and  some  authors  believe  that 
they  could  be  applied  at  the  secondary,   and  perhaps 
even  at  elementary  levels.     Some  studies  have  shown 
a  modest  correlation  between  student  ratings  of 
teachers  and  student  achievement,   but  questions 
about  their  validity  and  utility  limit  their 
acceptance . 

6.  Peer  Review.     In  this  process  the  teacher  is 
evaluated  by  a  committee  of  peers  who  examine  the 
whole  teaching  assignment.     Research  in  this  area 
shows  divergent  results  and  the  method  is 
unsuitable  for  evaluations  involving  personnel 
dec isions  . 

7.  Student  Achievement.     Since  the  ultimate  aim  of 
teaching   is  student  learning,   it  would  seem 
reasonable  to  assess  teacher  competency  or 
performance  by  measuring  student  achievement  on 
test  scores.     Studies  of  the  reliability  of  student 
test  scores  as  a  measure  of  teacher  effectiveness 
consistently  indicate  that  reliability  is  low 
(Brophy,   1973;    Rosenshine,  1970). 

8.  Faculty  Self  Evaluations.  Faculty  self  evaluation 
has  joined  other  sources  as  a  technique  in  teacher 
evaluation.  The  process  is  clearly  unsuitable  for 
accountability  or  personnel  decisions  but  may  have 
value  in  that  it  permits  the  teacher  to  assess  his 
or  her  own  strengths  and  weaknesses  and  may  promote 
teacher  professionalism.  (condensed  and 
paraphrased  from  Darling-Hammond  et  al.,  1984,  pp. 
303-308) 

The  discussion  above  suggests  that  techniques  available 
for  evaluation  serve  a  variety  of  purposes.     Some  seek  to 
measure  competence  by  testing  the  teacher,  others  rate 
performance  through  observational  and  interview  strategies, 
while  still  others  seek  to  measure  teacher  effectiveness  by 
measuring  the  outcomes  of  teaching.     The  low  levels  of 
reliability  of  most  of  these  measures  tend  to  suggest  that 
any  single  approach  for  assessing  competence,  performance. 
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or  effectiveness  is  unlikely  to  provide  a  sound  basis  for 
useful  teacher  evaluation. 

The   Impact  of  Evaluation  on  Student  Learning 

If  the  avowed  aim  of  evaluation  or  supervision  is  the 
improvement  of  instruction,   then  the  evaluation  of  teachers 
should  result  in  improved  learning  by  students  and  with 
current  high  levels  of  interest  in  evaluation,  one  would 
expect  to  find  many  studies  linking  evaluation  or 
supervision  with  student  performance.     However,  Acheson  and 
Gall   (1980,  p.   22)   note  that  they  were  not  able  to  find  any 
recent  research  on  student  effects  associated  with  clinical 
supervision,  and  searches  of  the  ERIC,  Dissertation 
Abstracts,  and  Psychological  Abstracts  indices  by  this 
researcher  failed  to  reveal  any  studies  more  recent  than 
those  documented  below. 

Burton   (1922,  p.  8)  cites  "a  pioneer  study"  carried  out 
in  1919  in  the  Detroit  Public  Schools  by  Courtis  and  Barns 
which  showed  that  the  results  in  geography  work,  when 
measured  with  a  standard  test,  were  "markedly  superior"  in 
schools  with  a  special  supervisor,  and  that  schools  with 
ordinary  supervision  were  superior  to  those  having  no 
supervision . 

Barr  et  al  .    (  1938,  pp.   807-816  )   cite  a  number  of 
controlled  studies  carried  out  in  Detroit  and  rural  Michigan 
in  the  1920s.     The  studies  showed  dramatic  differences  in 
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student  achievement  in  supervised  and  unsupervised  schools. 
One  study  showed  that  in  thirteen  school  functions,  the 
students  in  the  supervised  schools   "advanced  approximately 
194  per  cent  as  far  during  the  seven  months   ...  as  did  the 
children  with  whom  they  were  compared."     Another  study, 
carried  out  in  rural  Michigan,   using  a  variety  of 
standardized  tests  in  reading,   language,   spelling,  and 
mathematics  compared  the  gains  of  pupils  in  supervised  and 
unsupervised  schools.     During  the  study  period,  September 
1924  to  May  1925,  pupils  in  supervised  schools  showed  gains 
of  170.8  per  cent  of  a  normal  year  of  school  work  whereas 
pupils  in  unsupervised  schools  showed  gains  of  97  per  cent. 
Similar  studies  carried  out  in  Indiana  and  Louisiana  showed 
similar  results. 

Teacher  Effectiveness 

Tied  to  the  whole  process  of  teacher  evaluation  is  the 
guestion  of  what  is  successful  teaching.     For  several 
decades  researchers   (e.g.,   Barr,   1958;   Biddle  &  Elena,  1964; 
Bishop,   1981;  Gough,   Durflinger,   &  Hill,   1968;   Medley,  1979; 
Rosenshine  &   Berliner,   1978)  have  attempted  to  identify 
those  characteristics  of  teachers  that  make  them  successful. 
Hersh   (1981),   in  a  comprehensive  review  of  the  literature, 
defines  effectiveness  as  "student  academic  achievement  as 
measured  by  nationally  normed,   standardized  achievement 
tests"  and  goes  on  to  delineate  the  characteristics  of 
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effective  schools.     He  differentiates  between  two  sets  of 
conditions:     Social  Organization  and  Instruction  and 
Curriculum.     Under  these  broad  headings  he  lists  the 
attributes  which  pervade  effective  schools.     In  the  Social 
Organization  domain,  effective  schools  have  clear  academic 
and  social  behavior  goals,  order  and  discipline,  high 
expectations,  public  rewards  and  incentives,  administrative 
leadership,  community  support,  and  a  sense  of  pervasive 
caring  and  teacher  efficacy.     In  the  area  of  Instruction  and 
Curriculum  effective  schools  exhibit  high  academic  learning 
time,   frequent,  monitored  homework,   frequent  monitoring  of 
student  progress,  a  tightly  coupled  curriculum,  a  variety  of 
teaching  activities,   and  opportunities  for  student 
responsibility.     Hersh  indicates  that  while  these  factors 
individually  have  little  impact  on  achievement  as  measured 
by  standardized  tests,   together  they  create  a  "synergy  of 
effect"  wherein  the  whole  is  greater  than  the  sum  of  the 
parts  . 

Berliner   (1982)   identifies  similar  factors  as 
indicators  of  effective  classroom  instruction,  namely: 
allocated  time,  engaged  time,  opportunity  to  learn,  content 
coverage,  curriculum/test  congruence,  and  direct 
instruction.     Medley  and  Crook   (1982)   list  five  tasks  which 
distinguish  more  effective  teachers  from  less  effective 
teachers;   they  are:     maintaining  pupils'   task  involvement, 
teaching  in  large  groups,  minimizing  disruptive  behavior. 
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managing  small  group  activities,   and  supervising  pupil 
seatwork. 

The  elements  of  effective  teaching  that  result  in 

increased  achievement  on  standardized  tests  are  not 

necessarily  those  that  result  in  creativity,  problem  solving 

ability,  and  complex  cognitive  learning;   and  Centra  and 

Potter  (1980)  comment: 

Higher  order  skills   (e.g.   inferential  reasoning)  are 
not  particularly  likely  to  be  acquired   (and  certainly 
will  not  be  demonstrated)  by  students  whose  teachers 
ask  only  lower  order  questions.     (p.  285) 

In  addition,   research  by  Horwitz,   McKeachie  and  Kulik, 
Peterson,   and  Traub  et  al.,   cited  by  Darling-Hammond  et  al . 
(1984,  p.   296),   suggests  that  some  of  the  desirable 
affective  outcomes  of  education  result  from  teaching 
behaviors  that  are  different  from  those  deemed  useful  in 
improving  student  scores  on  standardized  tests. 

While  much  of  the  recent  research  has  focussed  upon  the 
teaching  behaviors  that  are  seen  as  effective  for  improving 
student  performance  on  standardized  tests,   it  can  be  seen 
that  other  teaching  strategies  leading  to  different  outcomes 
may  be  deemed  equally  desirable,  depending  upon  the 
educational  goals.     The  question  of  educational  goals  is 
therefore  critical  to  supervisory  or  evaluative  processes. 
Once  a  set  of  educational  goals  has  been  established,  then 
it  is  probable  that  the  research  on  effective  teaching  can 
serve  the  purposes  of  teacher  evaluation  for,   as  Townsend 
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(1984)   says,    "Many  of  the  typical  measures  of  teacher 
effectiveness  would  appear  to  lend  themselves  to  positive 
influence  through  the  use  of  appropriate  supervisory 
techniques"   (p.  18). 

Models  of  Teacher  Evaluation 

The  earliest  model  of  teacher  evaluation  was  the 
inspectoral  model  which  existed  from  the  early  1700s  until 
about  fifty  years  ago  (Burton  &   Brueckner,   1955).     In  the 
1920s,   the  improvement  of  instruction  emerged  as  the  major 
objective  of  supervision  and  the  period  from  1920  was 
dominated  by  two  models  of  supervision,   the  scientific  model 
and  the  democratic  model   (Lucio  &  McNeil,   1962;  Wiles  & 
Lovell,  1967). 

The  last  two  decades  have  seen  the  emergence  of  several 
similar  models,   such  as  the  Manatt  and  Redfern  models,  which 
involve  the  establishment  of  goals  and  some  level  of  teacher 
involvement  in  the  evaluative  process.     The  Manatt  "Mutual 
Benefit  Evaluation"   (Manatt,   Palmer,   &  Hidlebaugh,  1976) 
involves  the  establishment  of  criteria;   a  diagnostic 
evaluation  including  preobservat ion  conference,  objective 
data  gathering  and  a  pos tobservation  conference;  the 
establishment  of  job  targets;  and  a  reevaluat ion . 

Redfern   (1980)  developed  a  similar  model  based  on  the 
business  model  of  "management  by  objectives."     Like  the 
Manatt  model,   the  criteria  are  set  by  the  responsible  school 
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authority,   but   in  the   Pedfern  model,   the  teacher  and 
evaluator  jointly  determine  the  individual  objectives  and 
the  measurement  process  to  be  used. 

The  flexible,  multigoal  model  which  has  achieved 
prominence  today  is  the  Clinical  Supervision  model  developed 
by  Goldhammer   (1969)   and  Cogan  (1973). 

As  clinical  supervision  has  evolved  from  its 

beginnings,   so  a  variety  of  definitions  have  evolved; 

however,   Morris  Cogan 's   (  1973  )  definition  captures  the 

essence  of  the  concept: 

[Clinical  supervision  is]    the  rationale  and  practice 
designed  to  improve  the  teacher's  classroom 
performance.     It  takes  its  principal  data  from  the 
events  of  the  classroom.     The  analysis  of  these  data 
and  the  relationship  between  teacher  and  supervisor 
form  the  basis  of  the  program,  procedures,  and 
strategies  designed  to  improve  the  student's  learning 
by  improving  the  teacher's  classroom  behavior.     (p.  27) 

Goldhammer   (1969)  emphasizes  the  humanistic  nature  of 

the  process: 

Given  close  observation,  detailed  observational  data, 
face  to  face  interaction  between  the  supervisor  and 
teacher,   and  an  intensity  of  focus  that  binds  the  two 
together  in  an  intimate  professional  relationship,  the 
meaning  of  "clinical"   is  pretty  well  filled  out.  (p. 
54) 

Krajewski   (1982),    in  discussing  the  conceptual 
framework  of  clinical  supervision,   identifies  the  following 
eight  steps  as  Cogan 's   "cycle  of  supervision": 

1.  Establishing  the  teacher-supervisor  relationship. 

2.  Planning  with  the  teacher. 

3.  Planning  the  strategy  of  observation. 
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4.  Observing  instruction. 

5.  Analyzing  the  teaching-learning  process. 

6.  Planning  the  strategy  of  the  conference. 

7.  The  conference. 

8.  Renewed  planning. 

Acheson  and  Gall   (1980)    in  Techniques   in  the  Clinical 
Supervision  of  Teachers:     Preservice  and  Inservice 
Appl icat ions ,   have  expanded  upon  the  ideas  presented  by 
Cogan  and  Goldhammer  and  developed  a  model  which  is 
"interactive  rather  than  directive,  democratic  rather  than 
authoritarian,   teacher  centered  rather  than  supervisor 
centered "   ( p .  8 )  . 

A  model  for  a  teacher  evaluation  system  based  upon  the 
precepts  of  clinical  supervision  and  the  legal  ramifications 
of  improper  supervisory  practice  has  been  developed  by 
Acheson  (1982)  who  states  that  an  effective  teacher 
evaluation  program  should: 

1.  Contain  a  set  of  standards  of  performance  for 
teachers  . 

2.  Include  job  descriptions  for  teachers. 

3.  Require  that  evaluators  follows  a  cycle  of  pre- 
observation  conferences,  multiple  observations,  and 
post -observation  conferences  before  a  final 
evaluation  is  prepared. 

4.  Require  that  evaluators  have  training  and  skills  in 
conducting  classroom  observations. 

5.  Require  that  teachers  and  evaluators  cooperatively 
set  performance  goals. 
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6.  Provide  evidence  of  a  monitoring  or  management 
system  for  the  evaluators  themselves. 

7.  Require  that  a  program  of  assistance  be  made 
available  for  teachers  who  need  help  with  serious 
problems . 

If  tody's   (1983)   summary  of  Board  of  Reference  hearings 
and  the  recent  court  cases  in  the  United  States   (e.g.,  those 
reviewed  by  Zirkel  and  Cluckman,   1984)   tend  to  support 
Acheson's  model  and  emphasize  the  need  for  clearly  defined 
parameters  and  the  provision  for  affording  teachers  "all 
rights  to  natural  justice  and  due  process"  (Alberta 
Teachers'  Association,   1980,  p.  3). 

Teacher  Evaluation  in  Alberta 

While  there  are  large  number  of  articles  related  to 
teacher  evaluation  in  the  Alberta  literature   (e.g.,  Duncan, 
1980;   Keeler,   1980;   Reavis,   1980;   Stockman,   1982;   Tymko , 
1980),   the  researcher  uncovered  only  a  few  research  studies: 
Holdaway  and  Reikie   (1977),   Reikie   (1977),   Alberta  Education 
(1980),   Berg   (1983),   and  Townsend  (1984). 

Holdaway  and  Reikie   (1977)   and  Reikie   (1977)  carried 

out  a  study  into  the  "Practices  and  policies  involved  in  the 

formal  evaluation  of  teachers   in  Alberta."     The  study, 

drawing  its  data  from  questionnaires  completed  by 

superintendents,   focussed  on: 

1 .     The  categories  of  teachers  evaluated  and  the 
frequency  of  their  evaluation. 
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2.  The  personnel   involved   in  formally  evaluating 
teachers  . 

3.  The  formalized  procedures  and  criteria  used  in 
formally  evaluating  teachers. 

4.  The  uses  made  of  the  evaluations. 

5.  The  opinions  of  the  respondents  regarding 
contemplated  changes  in  the  existing  practices. 

Some  of  the  major  findings  of  the  study  were: 

1.  Tenured  teachers  were  rarely,  and  sometimes  never 
evaluated;  probationary  teachers  were  usually 
evaluated  only  once  a  year. 

2.  Superintendents  were  often  extensively  involved  in 
performing  teacher  evaluations;  however,   they  would 
prefer  greater  involvement  from  other  personnel, 
particularly  principals. 

3.  Standard  evaluation  forms  were  possessed  by  many 
systems  but  were  not  often  used. 

4.  Appeal  procedures  were  absent  in  many  jurisdictions 
but  where  such  procedures  were  in  place  they 
commonly  included  discussing  the  matter  with  the 
superintendent  or  evaluator  and/or  requesting  a 
reevalua tion  . 

5.  The  most  frequently  emphasized  criteria  were 
process  or  pressage  criteria. 

6.  Formal  evaluations  were  used  most  frequently  as  a 
basis  for  recommending  permanent  certification  and 
awarding  permanent  contracts.     They  were  also 
commonly  used  as  the  basis  for  dismissals,   and  less 
frequently,   as  a  basis  for  promotion. 

7.  Most  superintendents  were  not  contemplating  any 
changes  in  their  formial  evaluation  practices, 
(adapted  from  Reikie,   1977,   pp.   iv,  v) 

The  authors  also  noted  that  few  systems  had  clearly 

structured  evaluation  policies  and  practices,   that  practices 

across  the  province  were  inconsistent,  and  that  many 

jurisdictions  had  no  teacher  evaluation  policy  in  place. 
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The  Alberta   Education   (1980)   study,  entitled 
"Certificated  Education  Staff  Evaluation,"  was  essentially 
an  abbreviated  replication  of  the  1977  studies  and  its 
findings  indicate  that  a  number  of  changes  occurred  during 
the  three  intervening  years.     The  most  notable  changes  were: 

1.  Between  1977  and  1980  the  use  of  formal  evaluations 
for  the  purpose  of  improving  teacher  performance 
dramatically  increased   (18%   in  1977  and  83%  in 
1980)  . 

2.  An  increasing  number  of  teachers,   including  first 
year  teachers,  experienced  teachers  new  to  the 
system,   and  tenured  teachers,  were  being  evaluated. 

3.  Beginning  teachers  were  evaluated  more  frequently 
in  1980. 

4.  The  number  of  jurisdictions  reporting  that  they 
used  a  standardized  form  increased  from  18%   in  1977 
to  52%   in  1980.     (adapted  from  Alberta  Education, 
1980,  p.  17) 

The  Berg   (1983)   study  was  somewhat  similar  and 

questioned  a  sample  of  30  Alberta  superintendents  on  a 

number  of  items  related  to  the  policies  and  practices  of 

their  jurisdictions.     The  most  notable  findings  were: 

1.  That  an  increased  percentage  of  jurisdictions 
report  that  evaluation  policies  were  in  place  (52% 
in  1980,   78%   in  1983  )  . 

2.  That  about  one  half  of  respondents  stated  that 
their  jurisdiction  had  written  standards  of 
performance,   job  descriptions,  or  programs  of 
ass  is tance . 

3.  Most  jurisdictions  used  subjective  criteria  in 
their  evaluation  policies. 

4.  All  of  the  superintendents  surveyed  reported  that 
principals,  or  other  school  based  staff  were 
involved  in  formative  evaluations.     (adapted  from 
Berg,   1983,   p.  13) 
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The  Townsend   (1984)   study  was  quite  different  from  the 

preceding  studies.     It  sought  to  "investigate,   analyze,  and 

document  the  process  of  implementation  of  a  nev;  policy  of 

teacher  supervision  and  evaluation  in  the  five  secondary 

schools  of  the  Lethbridge  School  District  No.   51"   (p.  i). 

A  survey  of  teachers  '  and  supervisors  '  perceptions  of 
usual  and  ideal  supervisory  behavior  was  conducted  at 
the  beginning  and  end  of  the  study  period  and  other 
data  were  collected  through  the  administration  of 
questionnaires  and  frequent  observations  of  classrooms 
and  conferences. 

Analysis  of  the  survey  data  showed  that  teachers  and 
supervisors  perceived  a  change  in  supervisory  behavior 
towards  what  both  groups  considered  as  ideal 
supervisory  behavior.     Supervisors  reported  changed 
teacher  behaviors  especially  in  the  areas  of  classroom 
management,  planning,  and  increased  attention  to 
individual  students. 

Overall  the  results  of  the  study  showed  high  levels  of 
acceptance  and  participation,  high  levels  of 
satisfaction  with  the  process,   and  a  need  for  more 
training  and  more  involvement  of  district  office 
personnel   in  the  future.     (adapted  from  Townsend,  1984, 
pp.  i-iii) 


Conclusion 


This  review  of  the  literature  illustrated  the  diverse 
and  sometimes  contradictory  nature  of  the  research  on 
teacher  evaluation  and  traced  the  development  of  evaluation 
from  its  beginnings  as  an  autocratic,   inspectoral  process  to 
the  rather  more  humane,  multigoal,  democratic  interactive 
process  exemplified  by  Acheson's   "state  of  the  art"  model. 

The  dearth  of  research  on  teacher  evaluation  in  Alberta 
illustrates  the  need  for  more   information  on  current 
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practices,   particularly  with  respect  to  the  modern  models 
based  upon  the  clinical  supervision  process.     The  following 
chapters  in  this  dissertation  explore  the  demographics  of 
teacher  evaluation  in  Alberta,  and  examine  current  practices 
in  evaluation  with  respect  to  improvement  of  instruction  and 
due  process  criteria,  namely,   those  processes  common  to  the 
clinical  supervision  model. 
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CHAPTER  III 
DESIGN  AND  PROCEDURES 
Introduction 

To  gather  data  on  the  state  of  formal  evaluation  in 
Alberta,   this  study  involved  a  nuinber  of  steps: 

1.  A  three  part  questionnaire  (Appendix  A)  was 
developed  to  collect  data  related  to  formal 
evaluation  practices. 

2.  A  study  group  consisting  of  Alberta  school 
superintendents,  other  central  office  staff,  and 
principals  was  selected. 

3.  The  questionnaire  was  administered  to  the  study 
group . 

4.  Minimum  criteria  in  the  areas  of  "due  process"  and 
"improvement  of  instruction"   in  teacher  evaluation 
were  established  by  submitting  Section  3  of  the 
questionnaire  to  experts  for  adjudication. 

5.  Teacher  perceptions  of  appropriate  evaluation 
strategies  were  assessed  by  administering  Section  3 
of  the  questionnaire  to  a  sample  of  teachers. 

6.  Data  received  from  these  sources  were  processed  as 
described  later  in  this  chapter. 
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Development  of  the  Questionnaire 

The  initial  draft  of  the  questionnaire  that  provided 
the  data  base  for  the  study  was  developed  following  a  review 
of  the  literature  and  draws  on  the  models  used  by  Acheson 
(1980,   1981),   Alberta   Education   (1980),   Holdaway  (1977), 
Reikie   (1977),   and  Shinn   (1976).     These  models  were  adapted 
on  the  basis  of  the  researcher's  experience  in  teacher 
evaluation  in  Alberta  to  form  a  compact  questionnaire  that 
focused  specifically  on  the  issues  of  this  study. 

The  initial  draft  consisted  of  two  sections.  Section 
one  drew  heavily  upon  the  Alberta  Education   (1980),  Holdaway 
(1977),   and  Reikie   (1977)  models  and  was  designed  to  collect 
demographic  data  and  information  on  the  intended  purpose  of 
the  evaluation.     Section  two  was  based  upon  the  models 
developed  by  Acheson   (1980,   1981)   and  Shinn  (1976). 

The  Acheson/Shinn  model  used  32  descriptive  statements 
of  classroom  supervisory  activities  that  request  responses 
on  a  five  point  Likert  scale  ranging  from  "never"  to 
"always."     The  items  from  this  model  were  rephrased  to  a 
"first  person"   format;   a  number  of  questions  were 
amalgamated  and  a  few  items,   relating  mainly  to  due  process 
considerations,  were  added.     The  27   items  thus  generated 
were  formatted  to  elicit  a  yes/no  response. 

Seven  people  evaluated  this  draft:     three  experienced 
supervisors,   a  teacher,  a  professor  of  education,  a 
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researcher  in  teacher  evaluation,   and  a  lawyer  whose 
expertise  lies  in  the  educational  domain.     These  people  were 
asked  to  judge  the  clarity  of  the  descriptive  statements  and 
the  appropriateness  of  the  items  for  assessing  teacher 
evaluation  practices. 

As  a  result  of  the  input  received,   the  initial  draft 
was  revised.     The  first  section  was  expanded  to  include  data 
on  the  initiating  factors  in  the  evaluation  and  its  outcome, 
and  the  second  section  was  increased  to  37  items,  of  which 
26  were  adaptations  of  the  Acheson/Shinn  items. 

The  instrument  was  then  piloted  with  ten  principals 
from  the  Lakeland  School  District  #   5460   in  Alberta.  The 
principals  were  asked  to  respond  to  the  questionnaire  and  to 
critique  it.     On  the  basis  of  their  responses  a  few  minor 
wording  modifications  were  made  and  the  questionnaire  was 
reformatted  into  a  three  section  instrument.     Section  one 
collected  demographic  data  about  the  responding  evaluator. 
Section  two  addressed  the  "last  formal  teacher  evaluation 
carried  out"  and  collected  data  related  to  the  initiating 
factors,  purposes,   and  outcomes  of  the  evaluation.  Section 
three  was  the  modified  Acheson/Shinn  instrument  designed  to 
acquire  data  on  the  supervisory  activities  used  in  "the  last 
formal  teacher  evaluation  you  carried  out." 

The  revised  instrument  was  then  piloted  with  nine 
superintendents  from  North  Eastern  Alberta.  Superintendents 
were  asked  to  respond  to  the  questionnaire  and  to  critique 
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it.  Based  upon  their  input,  no  further  changes  were  made  to 
the  instrument. 

Appendix  A  contains  the  final  draft  of  the 
questionnaire  . 

Identification  of  the  Study  Group 

The  Holdaway   (1977)   study  showed  that  the  personnel 
primarily  involved  in  the  formal  evaluation  of  teachers  are 
the  superintendent,  other  central  office  staff,   and  the 
school  principal;   that  in  less  than  10%  of  school 
jurisdictions  are  vice-principals   "always  or  frequently" 
involved  and  that  other  categories  of  evaluator  (teachers. 
Alberta  Education  consultants,  etc.)  were  rarely,   if  ever, 
involved.     A  study  by  Berg   (1983)   indicates  that  in  1983, 
vice-principals  were  involved  in  4%  of  evaluations  requiring 
written  reports   (Berg,   1983,  p.  8). 

The  populations  surveyed  in  this  study  included  school 
principals   in  public  and  separate  schools   in  Alberta,  the 
superintendents  of  Alberta  school  jurisdictions,   and  other 
central  office  staff  who  were  members  of  the  Conference  of 
Alberta  School  Superintendents   (C.A.S.S.)   as  of  December 
1983.     Data  on  principals  and  superintendents  were  obtained 
from  Alberta  Education  and  data  on  other  central  office 
personnel  were  obtained  from  the  Conference  of  Alberta 
School  Superintendents. 
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Sampl ing 

Superintendents 

Alberta  Education  lists  147  school  jurisdictions  in  the 
province.     Of  these,   two  lie   in  the  City  of  Lloydminster ,  a 
community  straddling  the  Alberta/Saskatchewan  border. 
Lloydminster  school  jurisdictions  are  governed  by  the 
Saskatchewan  Department  of  Education  and  were  therefore 
excluded  from  the  study.     The  remaining  145  Alberta  school 
jurisdictions  are  served  by  106  superintendents. 
Questionnaires  were  sent  to  all  106  Alberta  superintendents. 

Other  Central  Office  Staff 

Central  office  staff  in  Alberta  school  jurisdictions 
can  be  categorized  into  two  groups: 

1.  Those  staff  members,  who  by  virtue  of  their  job 
description  or  contract,  are  full  members  of  the 
Alberta  Teachers'  Association   (A.T.A.)   and  are  thus 
governed  by  the  rules  of  the  A.T.A. 

2.  Those  staff  members  who,  by  virtue  of  their  job 
description  or  contract  are  specifically  excluded 
from  full  membership  in  the  Alberta  Teachers' 
Association  (although  they  may  be  associate 
members).     These  staff  members  are  eligible  for 
membership  in  the  Conference  of  Alberta  School 
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Superintendents,   and,    "a  very  large  percentage  of 
those  eligible  are  members"    (C.A.S.S.,   1984,   p.  1). 
Since  the  allegiances  of  the  two  groups  tend  to  be 
different,   and  since  the  only  centralized  listing  of  central 
office  staff  other  than  the  superintendent  is  the  C.A.S.S. 
membership  list,   the  researcher  selected  as  the  "other 
central  office  staff"  population,   those  staff  members  whose 
names  appeared  on  the  December  1983  C.A.S.S.  membership 
list . 

120   "other  central  office"  staff  members  were 
identified  from  the  list;  however,   telephone  contacts  with 
non-respondents  indicated  that  between  December  1983  and  May 
1984,   seven  of  that  number  had  left  the  position,   leaving  a 
total  population  of  113.     Questionnaires  were  sent  to  all 
113   "other  central  office"  staff. 

Pr inc  ipals 

A  pilot  of  the  questionnaire  with  ten  principals  in 
Northern  Alberta  yielded  only  a  60%  return  rate  with  no 
follow-up.     It  was  therefore  surmised  than  an  attempt  to 
survey  the  entire  population  would  yield  a  relatively  low 
return  rate  or  require  a  prohibitively  expensive  series  of 
follow-up  procedures. 

Since  a  low  return  rate  would  not  permit  generalization 
to  the  whole  population,   a  random  sampling  procedure  was 
used.     This  made  mail  and  telephone  follow-up  procedures 
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economically  viable,  resulted  in  a  higher  return  rate  and 
permitted  generalization  of  the  results  to  the  population 
within  the  limits  of  sampling  error. 

A  random  sample  of  principals  was  drawn  to  provide  a 
sufficiently  large  number  of  cases  to  allow  for  meaningful 
statistical  analysis.     The  sample,   consisting  of  25%  of  the 
population,  was  provided  by  the  Computer  Services  Branch  of 
Alberta  Education  and  included  the  names  of  347  principals 
selected  randomly  by  computer.     Questionnaires  were  sent  to 
all  347  principals. 

Distribution  of  the  Questionnaires 

During  the  last  week  of  April  1984,  a  package 
containing  a  questionnaire,   a  letter  of  introduction,   and  a 
self-addressed,   reply-paid  envelope  was  mailed  to  each  of 
the  administrators  identified  in  the  sample. 

During  the  third  week  of  May,   a  similar  package 
containing  a  questionnaire,  a  reminder  letter  and  a  self- 
addressed,   reply-paid  enveloped  was  mailed  to  non- 
respondents. 

Three  weeks  later,  non-respondents  were  mailed  a 
personalized  reminder  letter,   together  with  a  questionnaire 
and  a  self-addressed,   reply-paid  envelope. 

During  the  last  week  of  June,   the  remaining  non- 
respondents  were  telephoned  at  their  place  of  work  and, 
where  the  person  was  available,   s/he  was  asked  to  respond  to 
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the  survey.     Where  the  person  was  not  available,  a  message 
was  left  requesting  a  response. 

All  participants  were  advised  that  individual  responses 
would  remain  confidential  and  that  the  results  of  the  study 
would  be  disseminated  in  the  Fall. 

Establishment  of  Minimum  Criteria 
for  Satisfactory  Evaluation 

One  of  the  purposes  of  this  study  was  "To  analyze  the 
completeness  of  teacher  evaluation  procedures  used  by 
evaluators  in  Alberta  with  respect  to:      (1)   the  improvement 
of  instruction,   and   (b)  due  process  requirements."     To  do 
this  an  instrument   (hereafter  called  The  Modified 
Acheson/Shinn  Instrument)  was  developed  and  administered  to 
five  experts  in  the  area  of  due  process,   to  nine  experts  in 
the  area  of  the  improvement  of  instruction,   and  to  51 
teachers  who  provided  the  teacher  perspective  on  the 
instrument. 

Development  of  the  Instrument 

Section  three  of  the  final  draft  of  the  questionnaire 
(Appendix  A)  was  isolated  and  modified  by  replacing  the 
yes/no  response  section  with  a  four-point  scale  containing 
the  descriptors:     Essential   (high  priority).  Desirable 
(moderate  priority).   Acceptable   (low  priority),  and 
Unacceptable.     The  Modified  Acheson/Shinn  Instrument  was 
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mailed  to  those   individuals   identified  as  experts. 

Identification  of  Experts 

The  Deans  of   Education  of  Alberta's  Universities, 
Alberta  Education,   the  Conference  of  Alberta  School 
Superintendents,   the  Alberta  School  Trustees  Association, 
and  the  Alberta  Teachers'  Association  were  asked  to  nominate 
"acknowledged  experts"   in  the  field  of  teacher  evaluation. 
Responses  were  received  from  each  of  these  agencies  and 
experts  were  identified  in  the  areas  of  "due  process"  and 
"improvement  of  instruction." 

Due  Process 

The  following  people  were  identified  as  experts  in  the 
area  of  "due  process  in  teacher  evaluation": 

Judy  Anderson,    B.A.,  LL.B.,   LL.M.,   Lawyer  for  the 
Alberta  School  Trustees  Association. 

Richard  McNally,  LL.B.,   Lawyer  for  the  Alberta  School 
Trustees  Association. 

Mary  Jo  Williams,   Ph.D.,   Staff  Officer  for  the  Alberta 
Teachers'  Association. 

C.   P.  Clarke,   LL.B.,   Lawyer  for  the  Firm  Field  and 
Field,   Barristers  and  Solicitors  and  legal  counsel  for  the 
Alberta  Teachers'  Association. 

M.B.   Bielby,  LL.B.,   Lawyer  for  the  Firm  Field  and 
Field,    Barristers  and  Solicitors  and  legal  counsel   for  the 
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Alberta  Teachers'  Association. 

During  the   first  week  of  May  1984,   a  package  containing 
a  personal   letter,   the  Modified  Acheson/Shinn  Instrument, 
and  a  reply-paid,   self-addressed  envelope  was  mailed  to  each 
of  the  persons  identified  as  experts.     The  experts  were 
asked  to  review  the  instrument  and  to  rate  the  items  in 
terms  of  their  importance  in  assuring   "due  process"  in 
teacher  evaluation.     Responses  were  received  from  all  of  the 
experts  listed  above  and  no  follow-up  procedures  were 
required. 

Improvement  of  Instruction 

The  following  people  were  identified  as  experts  in  the 
domain  of  "teacher  evaluation  for  the  improvement  of 
instruction" : 

Dr.  David  Townsend,   Researcher,  Lethbridge  School 
District  #51. 

Dr.  Alex  Proudfoot,   Department  of  Teacher  Education  and 
Supervision,   University  of  Calgary. 

Dr.   Eugene  Ratsoy,   Department  of  Educational 
Administration,   University  of  Alberta. 

Myrna  L.  Greene,   Coordinator,   Education  Research 
Centre,   University  of  Lethbridge. 

Dr.  Laurie  Mireau,   Instructional  Processes  Consultant, 
Edmonton  Public  School  Board. 
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Dr.  Leroy  Sloan,   Assistant  Superintendent,  Educational 
Services,   County  of  Parkland. 

Dr.  W.   Unruh,   Professor,   Department  of  Teacher 
Education  and  Supervision,   University  of  Calgary. 

Dr.  A.   Boberg,   Assistant  Professor,   Department  of 
Educational  Policy  and  Administration  Studies,   University  of 
Calgary . 

Dr.   P.  Heyman,   Professor,   Department  of  Educational 
Policy  and  Administration  Studies,   University  of  Calgary. 

During  the  first  week  of  May  1984,  a  package  containing  a 
personal  letter.   The  Modified  Acheson/Shinn  Instrument,   and  a 
reply-paid,   self-addressed  envelope  was  mailed  to  each  of  the 
persons  identified  as  experts.     The  experts  were  asked  to 
review  the  instrument  and  to  rate  the  items  in  terms  of  their 
importance  to  "evaluation  for  the  improvement  of  instruction." 
Responses  were  received  from  all  of  the  experts  listed  above. 

Teachers  '   Perceptions  of  Evaluation  Procedures 

At  the  request  of  the  Planning  Services  Branch  of 
Alberta  Education,  a  small,   random  sample  of  teachers  was 
surveyed  to  assess  their  reaction  to  the  questions  asked  in 
The  Modified  Acheson/Shinn   Instrument.     The  sample  was 
provided  by  the  Computer  Services  Branch  of  Alberta 
Education  and  included  the  names  of  51   teachers  selected 
randomly  from  all  teachers  in  school  jurisdictions  in 
Alberta . 
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During   the   third  week  of  May  1984,   a  package  containing 
The  Modified  Acheson/Shinn   Instrument,   a  covering   letter  and 
a  reply-paid,   self-addressed  envelope  was  mailed  to  each 
teacher  identified  in  the  sample. 

A  follow-up  package  was  m.ailed  to  non-respondents  three 
weeks  later,  and  during  the  last  week  of  June,   the  remaining 
non-respondents  were  telephoned  at  their  place  of  work  and, 
where  the  person  was  available,   s/he  was  asked  to  respond  to 
the  survey.     Where  the  person  was  not  available,   a  message 
was  left  requesting  a  response. 

Data  Processing 

Data  from  the  questionnaires  were  entered  onto  keypunch 
cards  and  verified.     The  keypunch  cards  were  processed  in 
the  University  of  Alberta  computer  and  the  data  transferred 
to  disk.     Using  the  SPSSX  Sprint  package,  frequency 
distributions  and  means  were  generated  for  all  variables  and 
the  initial  printout  was  examined  for  apparent  anomalies. 
The  single  anomaly  discovered  was  compared  with  the  original 
data  and  corrected. 

The  expert  ratings  in  the  areas  of  due  process  and 
improvement  of  instruction,   and  the  teacher  ratings  on  the 
Acheson/Shinn  Instrument  were  summed  and  averaged.  Using 
the  criterion  that  the  subset  of  items  with  a  mean  score  of 
1.75  or  less  constituted  the  minimum  criteria  for 
satisfactory  evaluation,   three  subsets  of  the  original  37 
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items  were  generated: 

1.  The  due  process  subset  consisting  of   items  2,  10, 
11,    12,    20,   21,   22,    23,    24,    25,    26,    31,    32,    33,  34, 
3 5  ,   and  36  . 

2.  The  improvement  of  instruction  subset  consisting  of 
items  2,   3,   4,   5,   10,   11,   12,   13,   20,   21,   22,  and 
items  24  through  37  inclusive. 

3.  The  teacher  preference  subset  consisting  of  items 
3,  4,  5,  20,  25,  26,  27,  28,  and  items  30  through 
37  inclusive. 

Data  relating  to  these  subsets  were  entered  into  the 
computer . 

Cross  tabulation  tables  were  generated  to  compare  the 
responses  of  superintendents,  other  central  office  staff,  and 
principals  on  the  several  of  the  items  in  sections  1  and  2  of 
the  questionnaire  and  of  each  of  the  items  in  section  3. 

Three  by  three  analysis  of  variance  was  carried  out  to 
test  for  significant  differences  in  the  scores  on  Section  3 
of  the  questionnaire  by  category  of  evaluator 
(superintendent,  other  central  office,  or  principal)  and 
category  of  teacher  evaluated   (first  year,  experienced  but 
new  to  the  system,   tenured).     Two  way  analysis  of  variance 
was  carried  out  to  test  for  significant  differences  in  the 
scores  on  Section  3  of  the  questionnaire  with  respect  to  the 
responses  on  Section  2,  questions  6  through  10.     Two  way 
analysis  of  variance  was  used  to  test  the  significance  of 
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the  response  to  individual   items  on  Section  3  of  the 
questionnaire  with  respect  to  the  sum  of  the  other  items  in 
the  same  section. 

The  Pearson  Product  Moment  correlation  was  used  to  test 
the  significance  of  the  relationship  between  the  scores  on 
Section  3  and  demographic  variables. 
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CHAPTER  IV 
PRESENTATION  AND  ANALYSIS  OF  DATA 
Introduction 

This  chapter  presents  the  findings  of  the  study  as 
derived  from  the  instruments  used  to  gather  the  data. 
The  results  are  presented  in  six  sections: 

1.  Demographic  factors  relating  to  all  respondents. 

2.  The  demographics,   initiating  factors,  intended 
purposes,  and  outcomes  of  the  "last  formal  teacher 
evaluation  carried  out"  by  the  respondents. 

3.  The  evaluation  strategies  used  in  the  "last  formal 
teacher  evaluation  carried  out"  by  the  respondents. 

4.  The  establishment  of  minimum  criteria  for 
satisfactory  evaluation. 

5.  The  relationship  between  actual  practice  and  the 
minimum  criteria  for  satisfactory  evaluation. 

6.  The  relationships  between  demographic  factors  and 
the  score  on  the  Modified  Acheson/Shinn  Instrument. 

Demographic  Data 

Response  Rate 

Five  hundred  and  sixty  six  questionnaires  were  sent  to 
superintendents,  other  central  office  staff,   and  principals, 
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and  a  total  of  484  usable  responses  were  received,  resulting 
in  an  85.5%  return  rate  for  the  study  group. 

Of  the  106  superintendents  surveyed,   99  (93.4%) 
returned  usable  responses,   two   (1.9%)  wrote  letters 
indicating  their  reluctance  to  participate  in  the  research, 
and  one   (0.9%)   returned  a  blank  questionnaire. 

Other  central  office  staff  returned  97   (85.8%)  of  the 
116  questionnaires  sent  out  and  one   (0.9%)   letter  was 
received  declining  to  participate  in  the  study. 

Principals  responded  by  returning  288   (83.3%)  of  the 
347  questionnaires  mailed  out.     One   (0.3%)   letter  was 
received  declining  participation. 

Demographic  Factors 

Table  1   illustrates  the  extent  of  the  respondents' 
experience  in  teacher  evaluation.     Superintendents  ranged  in 
experience  from  zero  to  31  years,  other  central  office  staff 
from  zero  to  37  years,   and  principals  from  zero  to  30  years. 
As  might  be  expected,   superintendents  have,  on  average,  more 
experience  than  other  central  office  staff,  who  in  turn,  are 
more  experienced  than  principals.     Of  note  is  the  relatively 
high  proportion   (19.2%)  of  superintendents  with  twenty  or 
more  years  experience.     The  possible  ramifications  of  this 
are  discussed  later  in  the  chapter. 

Table  2  shows  the  number  of  university  credit  courses 
in  teacher  evaluation  completed  by  the  respondents.  A 
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TABLE  1  .     Years  of  Experience  as  a  Teacher  Evaluator 


Years 
of 

Experience 


1-5 


6-10 


11  -  15 


16  -  20 


Over  20 


Mean 


Superintendents 
N  =  99 


2.0% 


12.1% 


19.2% 


35.4% 


12.1% 


19.2% 


13.8  years 


Other  Central 
Office  Staff 
N  =  97 


3.1% 


13.6% 


38  .6% 


29.2% 


8.3% 


7.2% 


11.4  years 


Principals 
N  =  288 


8  .0% 


25  .7% 


31.3% 


17  .0% 


11  .1% 


6.9% 


9.3  years 
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TABLE  2.     Number  of  University  Credit  Courses  in  the 
Evaluation  of  Teachers  Completed 
by  the  Respondents 


Number 
of 

Courses  * 


5  or  more 


Mean  number 
of  courses 


Superintendents 
N  =  99 


28  .3% 


18.2% 


23.2% 


12.1% 


11.1% 


7.1% 


1  .86 


Other  Central 
Office  Staff 
N  =  97 


28  .9% 


15.5% 


24  .7% 


15.5% 


12.4% 


3.0% 


1  .76 


Pr inc  ipals 
N  =  288 


46.5% 


27.1% 


14.6% 


7.6% 


2.4% 


1  .7% 


1  .00 


*A  University  credit  course  means  a  six  credit  course  from 
an  Alberta  university  or  six  semester  hours  or  nine  term 
hours  of  coursework  from  U.S.  universities. 
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course  was  defined  as  a  full  course  from  an  Alberta 
university  or  six  semester  hours  or  nine  term  hours  of 
coursework  from  a  United  States  university.  Superintendents 
and  other  central  office  staff  reported  a  range  of  zero  to 
six  courses,   and  principals  reported  a  range  from  zero  to 
nine  courses.     Again,  as  might  be  expected,  superintendents 
had,  on  average,  more  university  training  than  other  central 
office  staff,  who  had  more  training  than  principals. 

Over  one  quarter  of  the  superintendents  and  other 
central  office  staff,  and  almost  half  of  the  principals 
responding  to  the  survey  indicate  that  they  had  no 
university  training  in  the  evaluation  of  teachers. 

Table  3  presents  the  data  on  the  time  spent  by 
respondents  on  inservice  training   in  teacher  evaluation 
during  the  last  five  years. 

Responses  from  superintendents  showed  a  range  of  in 
service  time  from  zero  to  1000  hours,  with  98  of  the  99 
responses  falling   in  the  range  from  zero  to  120  hours.  A 
telephone  conversation  with  the  superintendent  with  the  1000 
hour  response  revealed  that  teacher  evaluation  had  been  his 
particular  area  of  study  for  the  past  few  years  and  that  the 
1000  hours   (    .    .    .   "a  conservative  estimate")  included 
reading  and  research,   and  the  giving  and  receiving  of  in 
service  training.     Because  this  response  comprised  over  one 
third  of  the  total  time  spent  by  all  superintendents  on  in 
service  training   in  teacher  evaluation,  and  was  so  atypical. 
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TABLE  3.     Hours  of   In  Service  Training   in  Teacher 
Evaluation  Undertaken  by  Members  of  the  Study 
Group  During  the   Past  Five  Years 


Hours  of 
In  Service 
Training 

Superintendents 
N  =  98 

Other  Central 
Office  Staff 
N  =  97 

Pr inc  ipals 
N  =  288 

0 

26  .5% 

26  .8% 

26  .7% 

1-6 

17.3% 

9.3% 

20  .8% 

7-12 

18.4% 

14  .4% 

18.8% 

13  -  18 

3.1% 

5  .2% 

6.3% 

19  -  24 

5.1% 

18.6% 

10  .4% 

25  -  30 

13.3% 

5.2% 

6.9% 

31  -  36 

1  .0% 

2.1% 

0.3% 

37  -  42 

3.1% 

1  .0% 

1  .4% 

43  -  48 

3.1% 

1  .0% 

1  .0% 

48  -  54 

2  .0% 

3.1% 

2.8% 

55  or  more 

7.1% 

13.3% 

4.5% 

Mean  number 
of  hours  of 
In  Service 

18.5 

23  .4 

14  .9 
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Table  3  does  not   include  the  data  from  this  respondent. 

Other  central  office  staff  reported  a  range  of  in 
service  training  time  from  zero  to  120  hours  and  principals 
reported  a  range  of  zero  to  200  hours. 

The  median  for  superintendents  was  9.8  hours,   for  other 
central  office  staff,   10.5  hours,  and  for  principals,  7.1 
hours.     More  than  a  quarter  of  the  respondents  in  each  group 
reported  that  they  had  undertaken  no  in  service  training 
during  the  last  five  years. 

Data  presented  later  in  the  chapter  will  show  that 
other  central  office  staff  evaluate  more  teachers   (on  a  per 
capita  basis)   than  do  superintendents,  who  in  turn  evaluate 
more  teachers  than  do  principals.     This  pattern  is  reflected 
in  the  time  spent  on  in  service  training. 

The  study  group  perception  of  the  existence  of  a 
district  teacher  evaluation  policy  is  reported  in  Table  4. 

Previous  studies  in  Alberta,  which  reported  the  data  on 
the  presence  of  policies  by  jurisdiction  indicated  that  in 
1980,   52%  of  the  jurisdictions  had  a  teacher  evaluation 
policy  in  place   (Alberta  Education,   1980),   and  in  1983  a 
policy  was  in  place  in  78%  of  the  jurisdictions  (Berg, 
1983).     By  comparison,   80.8%  of  respondents  in  this  study 
indicated  the  presence  of  a  policy  in  their  jurisdiction. 

In  light  of  the  recent  Alberta  Education  (1984) 
regulation  requiring  that  school  boards  have  a  written 
policy  in  place,   it  is  of  interest  that  14.5%  of  respondents 
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TABLE  4.     Study  Group  Perception  of  the  Existence 
of  a  District  Teacher  Evaluation  Policy 


Pol  icy 
in 
Place 


Yes 


No 


Unsure 


No  Response 


Percentage  of 
Respondents 
N  =  484 


80  .8% 


14  .5% 


3.7% 


1  .0% 
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indicated  that  their  jurisdiction  had  no  written  policy  on 
teacher  evaluation. 

Table  5  reports  the  study  group's  assessment  of  the 
adequacy  of  the  time  they  have  available  for  teacher 
evaluation.     The  data  show  that  very  few  of  the  respondents 
considered  the  time  available  totally  adequate  and  more  than 
50%  of  the  473  respondents  considered  that  the  time 
available  was  inadequate  or  totally  inadequate. 

More  other  central  office  staff  indicated  that  the 
evaluation  time  available  to  them  was  adequate   (30.5%)  than 
superintendents   (14.6%)  or  principals   (20.6%).     This  may 
well  be  because  they  evaluate  more  teachers   (on  a  per  capita 
basis)   than  superintendents  or  principals,  and  teacher 
evaluation  is  generally  an  integral  part  of  the  job 
description. 

When  asked  to  rate  the  priority  of  teacher  evaluation 
in  relation  to  all  of  their  duties,  over  half  of  the  other 
central  office  staff  and  the  principals,  and  almost  three 
quarters  of  the  superintendents,   indicated   (Table  6)  that 
teacher  evaluation  was  a  high  or  very  high  priority  item. 
Of  respondents  who  carried  out  teacher  evaluations  during 
the  year,   89%  rated  teacher  evaluation  as  a  high  or  very 
high  priority. 

The  general  perception  among  a  majority  of  respondents 
is  that  teacher  evaluation  is  a  high  priority  item,   but  that 
they  have  a  less  than  an  adequate  amount  of  time  to  carry 
out  this  function. 
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TABLE  5.     Study  Group  Assessment  of  the  Adequacy 
of  the  Time  Available  for  Teacher  Evaluation 


Time 
Aval lable 

Superintendents 
N  =  96 

Other  Central 
Office  Staff 
N  =  92 

Pr inc  ipals 
N  =  285 

Totally 
Adequate 
1 

3.1% 

9.8% 

6.3% 

2 

11.5% 

20  .7% 

14  .4% 

3 

29  .2% 

25.0% 

25  .6% 

4 

38  .5% 

28  .3% 

35.1% 

Totally 
Inadequate 
5 

17  .7% 

16.3% 

18.6% 
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TABLE  6.     Study  Group  Perception  of  the  Priority 
of  Teacher  Evaluation 


Priority 

Superintendents 
N  =  96 

Other  Central 
Office  Staff 
N  =  92 

Principals 
N  =  285 

Very 
High 
1 

25  .8% 

16.1% 

15.8% 

2 

47  .4% 

40  .9% 

37  .2% 

3 

15.5% 

23  .7% 

28  .8% 

4 

5.2% 

8  .6% 

10  .5% 

Very 
Low 
5 

6.2% 

10  .8% 

7  .7% 
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Evaluations  Performed 

During   the  study  period,   respondents  reported  that  a 
total  of  4,924  teachers  were  evaluated.     Of  these,  1454 
(29.5%)  were  evaluated  by  superintendents,   1631   (33.1%)  by 
other  central  office  staff,   and  1839   (37.3%)   by  principals. 

Not  all  respondents  were  involved  in  teacher  evaluations 
during  the  study  period  but  71.7%  of  superintendents,   67%  of 
other  central  office  staff  and  73.3%  of  principals  reported 
that  they  had  evaluated  one  or  more  teachers .     Of  the 
respondents  who  did  evaluate  teachers,  superintendents 
averaged  20.8  teachers,  other  central  office  staff,  25.1 
teachers,  and  principals,   8.7  teachers. 

The  Last  Formal  Teacher  Evaluation  Carried  Out 

Demographic  Data 

Table  7   illustrates  the  patterns  of  formal  teacher 
evaluation  during  the  study  period  and  during  the  last  formal 
teacher  evaluation  carried  out  by  respondents.     (N.B.  A 
formal  evaluation  may  consist  of  a  series  of  conferences  and 
observations  leading  to  a  single  written  report.)  The 
patterns  are  somewhat  similar  but  respondents  show  a  higher 
percentage  of  evaluations  of  new  teachers  in  the  category  of 
"last  formal  evaluation."     Since  the  questionnaires  were 
administered  in  May,   the  last  month  for  making  decisions  of 
tenure  versus  termination  on  new  teachers,   it  is  probable 
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TABLE  7.     Patterns  of  Teacher  Evaluation 
During  the  Study  Period 


Teachers 
Evaluated 
During 
Survey  Period 
N  =  4924 


Teachers 
Evaluated 

During 
"Last  Formal 
Evaluation" 

N  =  342 


First  Year 
Teachers 


21.9% 


30  .1% 


Experienced 
Teachers 
New  to 
School/System 


17.3% 


18.1% 


Tenured 
Teachers 


60  .8% 


51  .8% 
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that  respondents  were  placing  a  higher  priority  on  the 
evaluation  of  first  year  teachers  during  this  period. 

Table  8  shows   the  distribution  of  classroom 
observations  among  grade  levels  for  the  last  formal 
evaluation.     A  total  347  teachers  were  formally  evaluated 
and  many  of  these  were  observed  on  several  occasions,   and  at 
different  grade  levels;   thus,   the  number  of  reported 
observations  exceeds  347. 

The  largest  single  group  of  teachers  evaluated  (31.4%) 
were  those  teaching  at  the  junior  high  school  level. 

Table  9  indicates  the  distribution  of  observations  among 
subject  categories  for  the  last  formal  evaluation,     A  total 
347  teachers  were  formally  evaluated  and  many  of  these  were 
observed  on  several  occasions,   and  in  different  subject 
areas;   thus,   the  number  of  reported  observations  exceeds  347. 

Almost  one  third  of  the  observations  carried  out  were 
in  language  arts  classes,  and  just  under  a  quarter  were  in 
math.     Social  studies  and  all  the  sciences  accounted  for 
another  quarter  of  the  observations,  and  the  remaining  20% 
were  categorized  as  "other," 

Physical  education,   French,   art,   and  music  predominated 
among  the  responses  in  the   "other"  category,  but  the  study 
group  reported  a  wide  range  of  subjects  observed  including: 
industrial  education,   religion,   remedial  education,  drama, 
family  life  education,  home  economics,   business  education. 
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TABLE  8.     Distribution  of  Observations 
Among  Grade  Levels 


Grade 
Level 


Percentage 
of  Reported 
Observations 
N  =  477 


ECS 


2.5 


1-3 


24  .1% 


4-6 


23  .1% 


7-9 


31  .4% 


10  -  12 


14  .7% 


Spec .  Ed 


4  .2% 


Total 


100  .0% 
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TABLE  9.     Distribution  of  Observations 
Among  Subject  Categories 


Subject 
Observed 

Percentage  of 

Reported 
Observations 
N  =  606 

Language  Arts 

31  .4% 

Math 

22  .4% 

Social 

16.7% 

Sc  iences 

9.4% 

Other 

20  .1% 

Total 

100  .0% 
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vocational  education,  living  skills,  guidance,  crafts,  health, 
beauty  culture,   law,  computer  education,   and  Ukrainian. 

Table  10  reports  the  number  of  separate  classroom  visits 
made  by  respondents  during  the  last  formal  teacher  evaluation. 
Superintendents  reported  a  range  of  one  to  seven  visits  with  a 
mean  of  2.2  visits,  other  central  office  staff  ranged  from  one 
to  nine  visits  and  averaged  2.7  visits,  and  principals  ranged 
from  one  to  nine  visits  and  averaged  3.4  visits.     Over  80%  of 
the  respondents  indicated  that  they  had  made  two  or  more 
visits,  and  over  half  made  three  or  more  visits  during  the 
last  formal  teacher  evaluation. 

The  time  spent  in  actual  classroom  observation  during  the 
last  formal  teacher  evaluation  is  shown  on  Table  11.  The 
total  observation  time  reported  by  the  respondents  ranged  from 
ten  minutes  to  fifteen  hours.     The  mean  number  of  minutes 
spent  in  observation  was  163.9;   the  median  was  116.8  minutes. 

Table  12  reports  the  study  group  perception  of  the  degree 
of  adherence  to  school  board  policy  during  the  last  formal 
teacher  evaluation.     Of  the  347  members  of  the  study  group  who 
evaluated  teachers,   11.2%  made  no  response  to  this  item.  This 
may  very  well  have  been  because  they  perceived  that  their 
district  had  no  policy.     Of  those  who  responded, 
superintendents  showed  the  greatest  rate   (44.1%)  and 
principals  the  least  rate   (29.0%)  of  following  policy  strictly 
to  the  letter. 
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TABLE  10.     Number  of  Classroom  Visits  Made  During 
the  Last  Formal  Teacher  Evaluation 


Number 

O  L 

Vi  si  ts 

Superintendents 

M     —  P 
IN     —  DO 

Other  Central 

ULLlCt5  OliaEL 

N  =  62 

Pr inc  ipals 

XT      _      1  Q7 

1 

26  .5% 

14.5% 

10  .7% 

2 

44.1% 

38  .7% 

22.8% 

3 

17.6% 

27.4% 

26.4% 

4 

7.4% 

9.7% 

18.3% 

5 

0% 

4  .8% 

10.2% 

6  or  more 

4  .4% 

4  .8% 

11  .6% 

Mean 

2 .2  visits 

2.7  visits 

3  .4  visits 
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TABLE  11.     Number  of  Minutes  Spent  in  Classroom 
Observation  During  the  Last  Formal 
Teacher  Evaluation 


Minutes  Spent 
in 

Observation 

Percentage  of 
Respondents 
N  =  347 

0-60 

17.3% 

61  -  120 

35  .4% 

120  -  180 

19.6% 

180  -  240 

13.8% 

240  -  300 

3.7% 

Over  300 

7.5% 

No  Response 

2.7% 

Mean  number  of  minutes  per  formal  evaluation 


163  .9 
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TABLE  12.     Study  Group  Perception  of  Their  Degree 
of  Adherence  to  School  Board  Policy  During  the 
Last  Formal  Teacher  Evaluation 


Degree  of 
Adherence 
to  Board 
Pol  icy 

Superintendents 
N  =  59 

Other  Central 
Office  Staff 
N  =  63 

Principals 
N  =  186 

Strictly  to 
the  letter 
1 

44  .1% 

41.3% 

29.0% 

2 

44  .1% 

52.4% 

49.5% 

3 

6.8% 

4  .7% 

16.1% 

4 

5.0% 

1  .6% 

3  .8% 

Not  at  all 
5 

0% 

0% 

1.6% 
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Table  13  reports  the  respondents'   rating  of  the 
performance  of  the  last  teacher  formally  evaluated. 
Although  almost  two  thirds  of  all  respondents  rated  the 
teachers  in  categories  one  and  two,  principals  rated  more 
teachers  in  these  categories  than  did  other  central  office 
staff  or  superintendents.     Superintendents  and  other  central 
office  staff  reported  a  much  higher  proportion  of  teachers 
whose  performance  was  rated  as  not  satisfactory. 

Since  it  is  commonly  the  task  of  the  superintendent  or 
other  central  office  staff   (rather  than  the  principal)  to 
make  the  final  decision  on  the  tenure  or  contract  status  of 
a  teacher,   it  seems  reasonable  that  they  would  place  greater 
emphasis  on  the  formal  evaluation  of  unsatisfactory  teachers 
than  would  principals.     One  might  also  speculate  that,  in 
the  interests  of  maintaining  harmony  and  a  collegial 
relationship  with  staff,  principals  might  avoid  the  onerous, 
and  often  contentious,   task  of  formally  evaluating  those 
teachers  whose  performance  they  perceived  as  unsatisfactory. 

Table  14  shows  the  primary  intended  purposes  of  the 
last  formal  teacher  evaluation  carried  out  by  the 
respondents.     Some  respondents  indicated  more  than  one 
primary  purpose  for  the  evaluation  so  the  number  of 
responses   (424)  exceeds  the  number  of  formal  evaluations 
reported   (347).     Other  central  office  staff  and  principals 
most  frequently  cited  improving  the  performance  of  the 
teacher  as  the  primary  purpose,  whereas  superintendents 
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TABLE  13.     Respondents'   Rating  of  the  Performance  of  the 
Last  Teacher  Formally  Evaluated 


Rating 

Superintendents 
N  =  69 

Other  Central 
Office  Staff 
N  =  65 

Principals 
N  =  204 

Excellent 
1 

18.8% 

13.8% 

32  .8% 

2 

33  .3% 

52  .3% 

38  .7% 

3 

23  .2% 

20  .0% 

22.1% 

4 

13.0% 

4  .6% 

5.9% 

Not 

Satisfactory 
5 

1 

11  .5% 

9.2% 

0.5% 
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TABLE  14.     Primary   Intended   Purpose  of  the  Last 
Formal  Evaluation 


Primary  Intended  Purpose  Percentage  of  Responses 

S  O  P  * 

N  =      (88)  (77)  (259) 


Improving  performance  of  teacher 

19.3% 

31  .2% 

29  .3% 

Awarding  a  permanent  contract 

26 

.1% 

28.6% 

12.7% 

Basis  for  retention  in  position 

19 

.3% 

13  .0% 

13.9% 

Recommending  permanent  certification 

14 

.8% 

7  .8% 

10  .4% 

Determining   inservice  needs 

5 

.7% 

5.2% 

7  .7% 

Basis  for  dismissal 

5 

.7% 

7.8% 

2.3% 

Basis  for  transfer 

1 

.1% 

0% 

3.5% 

Recording  the  performance  of  a 
resigning/retiring  teacher 

1 

.1% 

1  .3% 

1.9% 

Basis  for  promotion 

1 

.1% 

1.3% 

1.2% 

Other 

5 

.7% 

5.2% 

17  .0% 

*  S  =  Superintendents 

O  =  Other  Central  Office  Staff 
P  =  Principals 
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ranked  this  function  second  to  the  awarding  of  a  permanent 
contract.     The  dismissal  of  teachers  was  a  primary  purpose 
for  the  last  evaluation  for  4.1%  of  all  respondents; 
however,   it  was  more  often  the  primary  purpose  with  other 
central  office  staff   (7.8%)   than  with  superintendents  (5.7%) 
or  principals  (2.3%). 

In  the  "other"  category,   the  majority  of  responses 
indicated  that  the  primary  purpose  of  the  evaluation  was  to 
comply  with  a  board  policy;  however,  a  few  respondents 
indicated  that  their  primary  purpose  was  to  provide 
documentation  for  the  Edwin  Parr  Teacher  of  the  Year  Award, 
an  award  presented  annually  by  the  Alberta  School  Trustees 
Association . 

Table  15  reports  the  factors  involved   in  initiating  the 
evaluation.     Some  respondents  indicated  more  than  one 
initiating  factor  for  the  evaluation  so  the  total  number  of 
responses   (386)  exceeds  the  number  of  formal  evaluations 
reported   (  347  )  . 

Almost  two  thirds  of  the  responses  indicated  that  the 
evaluation  was  a  routine  evaluation  as  required  by  school 
board  policy.     The  responses  were  similar  across  all  three 
evaluator  categories  although  superintendents  showed  a 
slightly  greater  emphasis  on  the  need  for  documentation. 
Only  11.5%  of  the  respondents  indicated  that  the  evaluation 
was  initiated  in  response  to  evaluator  concern  or  parent 
complaints . 
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TABLE  15.     Primary  Condition   Involved   in  Initiating 
the  Last  Formal  Teacher  Evaluation 


Initiating  Factor  Percentage  of  Responses 

S  O  P  * 


N  = 

(82) 

(73) 

(231) 

District  policy:     Routine  evaluation 

57  .3% 

60  .3% 

66  .7% 

Need  for  documentation 

17.1% 

13  .7% 

11  .7% 

Teacher  request 

9  .8% 

9.6% 

5.6% 

Concerns  after  informal  observations 

7.3% 

5.5% 

6.5% 

Parental  complaint 

3.7% 

6  .8% 

5.2% 

Other 

4.9% 

4.1% 

4.3% 

*  S  =  Superintendents 

O  =  Other  Central  Office  Staff 
P  =  Principals 
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"Other"   initiating  factors  included  variations  on  the 
need  for  documentation,   and   the   Edwin  Parr  Award. 

Table  16  reports  the  outcome  of  the  last  formal  teacher 
evaluation.     Some  respondents  indicated  more  than  one 
outcome  for  the  evaluation  so  the  total  number  of  responses 
(432)  exceed  the  number  of  formal  evaluations  reported 
(347).     The  most  frequently  reported  outcome  by  all 
evaluator  categories  was  that  the  teacher  would  continue  in 
his  or  her  present  position.     The  awarding  of  a  permanent 
contract,   recommendation  for  permanent  certification,  and 
the  development  of  an  in  service  training  plan  were  other 
commonly  cited  responses.     Only  three  teachers   (0.7%  of  all 
responses)  challenged  the  evaluation  report. 

Teacher  dismissal  was  reported  as  the  outcome   in  4.8% 
of  all  responses,   but  as  might  be  expected,  evaluations 
conducted  by  superintendents  resulted  in  a  higher  dismissal 
rate   (9.0%)   than  those  conducted  by  other  central  office 
staff   (4.6%)  or  principals   (3.5%).     Responses  in  the  "other" 
category  included  the  filing  of  reports   in  the  teacher's 
personal  file  and  recommendation  for  the  Edwin  Parr  Award. 

In  summary,   the  majority  of  the  last  formal  teacher 
evaluations  carried  out  by  the  study  group  were   initiated  as 
routine  evaluations  according  to  school  board  policy;  their 
primary  purposes  were:     improving  the  performance  of  the 
teacher,   awarding  a  permanent  contract,  providing  a  basis 
for  retention  in  position,   and  recommending  for 
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TABLE  16.     Outcome  of  the  Last  Formal  Teacher  Evaluation 


Outcome  of  the  Evaluation  Percentage  of  Responses 

S  O  P  * 


The  teacher  was/will: 

N  =  (87) 

(87) 

(258  ) 

continue  in  present  position 

35.6% 

36 

.7% 

45 

.7% 

receive  a  permanent  contract 

20  .7% 

20 

.7% 

11 

.2% 

be  recommended  for  permanent 

14  .9% 

8 

.0% 

12 

.4% 

certi  f icat ion 

assisted  with  an  inservice  plan 

4.6% 

14 

.9% 

10 

.1% 

dismissed 

9.0% 

4 

.6% 

3 

.5% 

seen  by  another  evaluator 

4  .6% 

4 

.6% 

4 

.7% 

promoted 

0% 

3 

,4% 

1 

.6% 

challenge  the  evaluation  report 

0% 

1 

.1% 

0 

.8% 

Other  10.3%         5.7%  10.1% 


*  S  =  Superintendents 

0  =  Other  Central  Office  Staff 
P  =  Principals 
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permanent  certification.     As  a  result  of  the  evaluation, 
most  teachers  were  retained  in  their  present  position, 
received  a  permanent  contract  or  were  recommended  for 
permanent  certification. 

Establishment  of  Minimum  Criteria 
for  Satisfactory  Evaluation 

Due  Process  Criteria 

Copies  of  the  modified  Acheson/Shinn  instrument  were 
sent  to  the  five  people   identified  as  experts  in  the  area  of 
due  process  in  teacher  evaluation.     The  experts  were  asked 
to  rate  each  item  as   (1)   Essential,    (2)   Desirable,  (3) 
Acceptable,   or   (4)   Not  Acceptable. 

Three  of  the  experts  returned  usable  responses,  another 
replied  with  a  letter  declining  to  respond,   and  the  other 
addressed  the  items  in  more  general  terms. 

The  instrument  was  scored  with  items  rated  as  essential 
receiving  one  point;  desirable,   two  points;  acceptable, 
three  points;   and  not  acceptable,   four  points.     Table  17 
shows  the  ranked  mean  scores  of  the  ratings. 

Items  receiving  a  mean  score  between  1.00  and  1.75  were 

rated  as  essential  to  assuring  due  process  in  formal  teacher 

evaluation.     Those  items  were: 

Item  2  .       Standards  of  performance  had  been  developed 
for  the  teacher   (score  1.66). 

Item  10.     I  recorded  data  on  the  teacher's  questions 
for  later  analysis   (score  1.66). 


77 


TABLE  17.     Experts'   Ratings  of  Items  on  the  Modified 
Acheson/Shinn  Instrument 


Item  #  Descriptor 

I 

P 

T* 

25 

Met  with  teacher  after  observation 

1 

.00 

1  .00 

1.16 

20 

Observed  for  at  least  30  minutes 

1 

.00 

1  .00 

1.43 

35 

Teacher  received  a  copy  of  report 

1 

.13 

1.33 

1.10 

32 

Developed  recommendations 

1 

.50 

1  .00 

1.39 

34 

Developed  a  plan  of  assistance 

1 

.50 

1  .00 

1  .47 

36 

Teacher  could  respond  in  writing 

1 

.13 

1  .66 

1  .32 

33 

Developed  recommendations  with  teacher 

1 

.25 

1.66 

1.23 

37 

Appeal  procedure  available 

1  .00 

2  .00 

1  .23 

26 

Solicited  teacher  analysis  of  data 

1 

.50 

1.66 

1  .29 

31 

Recommended  resources 

1 

.50 

1  .66 

1.45 

30 

Gave  praise  and  encouragement 

1 

.25 

2.33 

1.32 

27 

Encouraged  teacher  inferences  &  opinions 

1 

.38 

2.33 

1  .19 

28 

Encouraged  consideration  of  alternatives 

1 

.29 

2  .00 

1.71 

22 

Compared  with  Alberta  Program  of  Studies 

1 

.75 

1  .33 

2  .03 

24 

Examined  student  work 

1 

.63 

1  .66 

1  .87 

2 

Standards  of  performance  developed 

1 

.57 

1  .66 

2  .00 

3 

Met  prior  to  observation 

1 

.75 

2  .00 

1  .52 

12 

Recorded  data  on  on-task  behavior 

1 

.75 

1  .66 

2.00 

10 

Recorded  data  on  teacher  questions 

1.75 

1  .66 

2.03 

21 

Examined  teacher's  planbook 

1 

.63 

1  .66 

2.19 

5 

Found  teacher  concerns 

1 

.38 

2.66 

1  .52 

1 

Had  written  job  description 

1  .80 

2  .00 

1  .76 

11 

Recorded  data  on  student  responses 

1 

.75 

1.66 

2  .30 

4 

Found  teacher  expectations  of  students 

1 

.75 

2  .50 

1  .48 

29 

Listened  more  than  I  talked 

1 

.43 

2  .66 

1  .87 

23 

Examined  record  of  student  achievement 

2 

.13 

1.66 

2  .19 

13 

Recorded  data  on  patterns  of  response 

1 

.75 

2  .00 

2  .38 

7 

Teacher  chose  observational  strategies 

1  .88 

3  .00 

1  .77 

8 

Recorded  impressions  about  classroom 

2 

.25 

2  .33 

2  .20 

19 

Recorded  student/teacher  interactions 

1 

.88 

2  .66 

2  .33 

16 

Recorded  data  on  student  movement 

2 

.50 

2.33 

2.53 

9 

Made  notes  of  classroom  dialogue 

2 

.50 

2.33 

2  .53 

15 

Recorded  data  on  teacher  movement 

2.35 

2.66 

2  .50 

6 

Suggested  observational  techniques 

2 

.17 

3  .00 

2  .00 

18 

Recorded  data  on  specific  student 

3.00 

2  .66 

2.93 

17 

Made  video  recordings 

2 

.69 

3  .33 

3.14 

14 

Made  audio  recordings 

2 

.88 

3.33 

2  .97 

*I  =  Mean  score  of  ratings  by  experts  in  improvement  of 

instruction;  P  =  Mean  score  of  ratings  by  experts  in 
due  process;   T  =  Mean  score  of  teacher  ratings 


Items  rated  as  "essential"  have  a  mean  score  between  1.00  and 
1  .75 
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Item  11.  I  recorded  data  on  students'  responses  to  the 
teacher's  questions   (score  1.66). 

Item  12.     I  recorded  data  on  student  on  task  behavior 
( score  1.66). 

Item  20 .     I  remained  in  the  classroom  for  at  least  30 
minutes  when  I  visited   (score  1.00). 

Item  21.     I  examined  the  teacher's  planbook  (score 
1.66). 

Item  22.     I  specifically  checked  that  the  program  was 
in  accord  with  the  Alberta  Program  of  Studies  (score 
1  .33)  . 

Item  23.     I  examined  the  record  of  student  achievement 
( score  1.66)  . 

Item  24.     I  examined  student  work   (score  1.66). 

Item  25.  I  met  with  the  teacher  after  the  observations 
(score  1.00)  . 

Item  26.     I  solicited  the  teacher's  analysis  of  the 
data   ( score  1.66). 

Item  31.     I  recommended  resources  such  as  books  and 
training  programs  which  dealt  with  areas  in  which  the 
teacher  wished  to  improve   (score  1.66). 

Item  32.     I  developed  recommendations  to  assist  the 
teacher  improve  instruction   (score  1.00). 

Item  33.     I  developed  recommendations  in  cooperation 
with  the  teacher   (score  1.66). 

Item  34.     I  developed  a  plan  of  assistance  for  the 
teacher  as  an  outcome  of  the  evaluation   (score  1.00). 

Item  35.     I  gave  the  teacher  a  copy  of  the  evaluation 
report  (score  1.33). 

Item  36.     The  teacher  was  given  the  opportunity  to 
respond  in  writing  to  the  report   (score  1.66). 

In  essence,   the  items  related  to  due  process  suggest 
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1.  Standards  of  performance  must  be  developed  for  the 
teacher. 

2.  A  variety  of  objective  data  must  be  collected. 

3.  A  post  conference  must  be  held  with  the  teacher  to 
discuss  the  data  and  develop  recommendations  for 
improvement . 

4.  A  plan  of  assistance  must  be  developed  for  the 
teacher . 

5.  The  teacher  must  be  given  a  copy  of  the  evaluation 
report  and  have  the  opportunity  to  respond  in 
writing  . 

Improvement  of  Instruction  Criteria 

Copies  of  the  modified  Acheson/Shinn  instrument  were 
sent  to  the  nine  people  identified  as  experts  in  the  area  of 
the  improvement  of  instruction  in  formal  teacher  evaluation. 
The  experts  were  asked  to  rate  each  item  as   (1)  Essential, 
(2)   Desirable,    (3)   Acceptable,  or   (4)   Not  Acceptable. 

All  of  the  experts  returned  usable  responses;  however, 
one  reply  was  received  too  late  to  be  included  in  the  data 
processing . 

The  instrument  was  scored  with  items  rated  as  essential 
receiving  one  point;  desirable,   two  points;  acceptable, 
three  points;   and  not  acceptable,   four  points.     Table  17 
shows  the  ranked  mean  scores  of  the  ratings. 

Items  receiving  a  mean  score  between  1.00  and  1.75  were 


80 


rated  as  essential  to  the  improvement  of  instruction  in 

formal  teacher  evaluation.     Those   items  were: 

Item  2.       Standards  of  performance  had  been  developed 
for  the  teacher   (score  1.57). 

Item  3.       I  met  with  the  teacher  prior  to  the 
observation   (score  1.75). 

Item  4.  I  found  out  what  the  teacher  expected  the 
students  to  be  doing  during  the  visit   (score  1.75). 

Item  5 .       I  found  out  what  concerns  or  problems  the 
teacher  had   (score  1.38). 

Item  10.     I  recorded  data  on  the  teacher's  questions 
for  later  analysis   (score  1.75). 

Item  11.  I  recorded  data  on  students'  responses  to  the 
teacher's  questions   (score  1.75). 

Item  12.     I  recorded  data  on  student  on  task  behavior 
( score  1.75). 

Item  20.     I  remained   in  the  classroom  for  at  least  30 
minutes  when  I  visited   (score  1.00). 

Item  21.  I  examined  the  teacher's  planbook  (score 
1  .63)  . 

Item  22.  I  specifically  checked  that  the  program  was 

in  accord  with  the  Alberta  Program  of  Studies  (score 
1.75). 

Item  24.  I  examined  student  work   (score  1.63). 

Item  25.  I  met  with  the  teacher  after  the  observations 
(score  1 .00) . 

Item  26.     I  solicited  the  teacher's  analysis  of  the 
data   ( score  1.50). 

Item  27.     I  encouraged  the  teacher  to  make  inferences 
and  to  express  feelings  and  opinions  about  the 
observational  data   (score  1.38). 

Item  28.     I  encouraged  the  teacher  to  consider 
alternative  teaching  techniques  and  explanations  of 
classroom  events   (score  1.29). 
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Item  29.  I  listened  more  than  I  talked  during  the  post 
observation  conference   (score  1.43). 

Item  30.     I  gave  praise  and  encouragement  for  specific 
growth  in  the  teacher's  teaching  skills   (score  1.25). 

Item  31 .     I  recommended  resources  such  as  books  and 
training  programs  which  dealt  with  areas  in  which  the 
teacher  wished  to  improve   (score  1.50). 

Item  32.     I  developed  recommendations  to  assist  the 
teacher  improve  instruction   (score  1.50). 

Item  33.     I  developed  recommendations  in  cooperation 
with  the  teacher   (score  1.25). 

Item  34.     I  developed  a  plan  of  assistance  for  the 
teacher  as  an  outcome  of  the  evaluation   (score  1.50). 

Item  35.     I  gave  the  teacher  a  copy  of  the  evaluation 
( score  1.13) . 

Item  36.     The  teacher  was  given  the  opportunity  to 
respond  in  writing  to  the  report   (score  1.13). 

Item  37.     The  teacher  had  recourse  to  a  formal  appeal 
procedure  if  he/she  challenged  the  report   (score  1.00). 

The  25   items  identified  as  essential  to  the  improvement 
of  instruction  generally  overlap  the  items  required  to  meet 
due  process  requirements,   the  major  difference  being  the 
rather  more  teacher-centered  nature  of  the  improvement  of 
instruction  subset.     In  essence,   to  meet  the  minimum 
criteria  for  the  improvement  of  instruction,   the  minimum 
criteria  to  assure  due  process  must  be  met   (with  the 
exception  of   Item  23,   the  examination  of  the  record  of 
student  achievement)  and: 

1.  A  preconf erence  must  be  held  to  discover  what  the 
teacher  expected  the  students  to  be  doing,  and  to 
discern  what  problems  or  concerns  the  teacher  had. 
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2.  During  the  post  conference,  greater  teacher  input 
must  be  sought  and  appropriate  praise  and 
encouragement  given. 

3.  A  formal  appeal  procedure  should  be  available  if 
the  teacher  challenged  the  report. 

Teacher  Perceptions 

Copies  of  the  modified  Acheson/Shinn  instrument  were 
sent  to  51  teachers  to  determine  teacher  perceptions  of  the 
essential  processes  of  formal  teacher  evaluation.  The 
teachers  were  asked  to  rate  each  item  as   (1)   Essential,  (2) 
Desirable,    (3)   Acceptable,  or   (4)   Not  Acceptable. 

Thirty  two  of  the  teachers   (62.7%)   returned  usable 
responses . 

The  instrument  was  scored  with  items  rated  as  essential 
receiving  one  point;  desirable,   two  points;  acceptable, 
three  points;  and  not  acceptable,   four  points.     Table  17 
shows  the  ranked  mean  scores  of  the  ratings. 

Items  receiving  a  mean  score  between  1.00  and  1.75  were 
rated  as  essential  in  formal  teacher  evaluation.  Those 
items  were: 

Item  3.       I  met  with  the  teacher  prior  to  the 
observation   (score  1.52). 

Item  4 .  I  found  out  what  the  teacher  expected  the 
students  to  be  doing  during  the  visit   (score  1.48). 

Item  5.     I  found  out  what  concerns  or  problems  the 
teacher  had   (score  1.52). 
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Item  20.     I  remained   in  the  classroom  for  at  least  30 
minutes  when   I  visited   (score  1»43). 

Item  25.  I  met  with  the  teacher  after  the  observations 
( score  1.16). 

Item  26.     I  solicited  the  teacher's  analysis  of  the 
data   { score  1.29). 

Item  27.     I  encouraged  the  teacher  to  make  inferences 
and  to  express  feelings  and  opinions  about  the 
observational  data   (score  1.19). 

Item  28.     I  encouraged  the  teacher  to  consider 
alternative  teaching  techniques  and  explanations  of 
classroom  events   (score  1.71). 

Item  30.     I  gave  praise  and  encouragement  for  specific 
growth  in  the  teacher's  teaching  skills   (score  1.32). 

Item  31.     I  recommended  resources  such  as  books  and 
training  programs  which  dealt  with  areas  in  which  the 
teacher  wished  to  improve   (score  1.45). 

Item  32.     I  developed  recommendations  to  assist  the 
teacher  improve  instruction   (score  1.39). 

Item  33.     I  developed  recommendations  in  cooperation 
with  the  teacher  (score  1.23). 

Item  34,     I  developed  a  plan  of  assistance  for  the 
teacher  as  an  outcome  of  the  evaluation   (score  1.47). 

Item  35.     I  gave  the  teacher  a  copy  of  the  evaluation 
report   (score  1.10). 

Item  36.     The  teacher  was  given  the  opportunity  to 
respond  in  writing  to  the  report   (score  1.32). 

Item  37.     The  teacher  had  recourse  to  a  formal  appeal 
procedure  if  he/she  challenged  the  report   (score  1.32). 

The  sixteen  items  identified  by  teachers  as  essential 

form  a  subset  of  the  improvement  of  instruction  items.  The 

most  notable  differences,  however,   fall  in  the  areas  related 

to  the  establishment  of  standards  and  the  collection  of 

objective  data.     Teachers  rate  all  of  these  items  lower  than 
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either  of  the  groups  of  experts,   and  NO  objective  data 
collection  strategies  were  rated  as  essential  by  teachers. 
In  addition,   teachers  rated  the  teacher  input  items  (items 
4,   5,   27,   and  28)  higher  than  the  other  groups. 

This  pattern  of  responses  by  teachers  suggests  to  the 
researcher  that  teachers  are  anxious  about  the  purposes  of 
formal  teacher  evaluation,   and,  notwithstanding  the  data 
presented  earlier,  perceive  the  process  as  punitive.  Thus, 
they  tend  to  rate  those  items  related  to  the  establishment 
of  standards  and  the  collection  of  objective  data  (essential 
components  of  any  dismissal  process)   lower  than  the  experts. 

Actual  Evaluative  Practices  Used  by  the  Study  Group 

Section  three  of  the  gues tionna ire  was  the  Modified 
Acheson/Shinn  Instrument  designed  to  acquire  data  on  the 
supervisory  activities  used  in  the  last  formal  teacher 
evaluation.     Respondents  were  asked  to  respond   "Yes"  or  "No" 
to  the  items. 

To  ensure  that  the  instrument  was,   in  fact,   a  scale 
with  no  anomalous  items,  each  item  was  examined  with  respect 
to  the  total  score  and  an  analysis  of  variance  was  carried 
out  to  compare  the  difference  between  "yes"  and  "no" 
responses.     For  all   items,   the  mean  score  achieved  by  "yes" 
respondents  was  higher  than  that  achieved  by  "no" 
respondents  and,  with  the  exception  of  items  14  and  17,  the 
difference  was  significant  well  beyond  the   .01   level.  Items 
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14  and  17  had  so  few  "yes"    (six  to  itein  14,   one  to  item  17) 
responses  that,   although  the  mean  for  the  "yes"  response  was 
greater  than  for  the  "no"  response  in  both  cases,  the 
difference  was  not  statistically  significant.     It  was 
therefore  concluded  that  the  instrument  was  a  scale  and  that 
there  were  no  anomalous  items. 

Table  18  reports,   in  rank  order,   the  percentage  of 
superintendents  who  responded  "yes"   to  each  of  the  items  on 
the  Modified  Acheson/Shinn  Instrument.     Similarly,   Table  19 
reports  the  data  on  other  central  office  staff  responses, 
and  Table  20,   the  principals'  data.     Table  21  reports  the 
consolidated  data  for  all  respondents  and  Table  22  presents 
a  comparative  ranking  of  "yes"  responses. 

Noteworthy  differences  occurred  on  ten  items: 

For  Item  3   (I  met  with  the  teacher  prior  to  the 
observation),   just  less  than  half  the  superintendents,  over 
three  quarters  of  the  other  central  office  staff,   and  86.7% 
of  principals  reported  a  "yes"  response. 

As  might  be  expected,  an  appeal  procedure   (Item  37)  was 
more  readily  available  when  the  evaluation  was  carried  out 
by  a  principal   (91.5%)   than  by  other  central  office  staff 
(87.7%)  or  by  the  superintendent  (69%). 

Superintendents  and  other  central  office  staff  ranked 
higher  than  principals  in  recording  student/teacher 
interactions   (Item  19),   recording   impressions  about  the 
classroom  (Item  8),   the  examination  of  students'  work  (Item 
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TABLE  18.     Percentage  of  Superintendents  Who   Responded  "Yes" 
to  Items  on  the  Modified  Acheson/Shinn  Instrument 


Percentage 

of  "Yes" 
Responses 


Item 

#                        Descriptor                                  n  = 

71 

Rank 

25 

Met  with  teacher  after  observation 

98 

.6% 

JL 

20 

Observed  for  at  least  30  minutes 

95 

.8% 

2 

19 

Recorded  student/teacher  interactions 

94 

.4% 

3 

8 

Recorded   impressions  about  classroom 

93 

.0% 

5 

24 

Examined  student  work 

93 

.0% 

5 

35 

Teacher  received  a  copy  of  report 

93.0% 

5 

21 

Examined  teacher's  planbooks 

91 

.5% 

7 

5 

Found  teacher  concerns 

90 

.1% 

8 

23 

Examined  record  of  student  achievement 

84 

.5% 

10 

30 

Gave  praise  and  encouragement 

84 

.5% 

10 

32 

Developed  recommendations 

84 

.5% 

10 

12 

Recorded  data  on  on-task  behavior 

81 

.7% 

12 

4 

Found  teacher  expectations  of  students 

78 

.9% 

13 

27 

Encouraged  teacher  inferences  &  opinions 

77 

.5% 

14 

.5 

36 

Teacher  could  respond  in  writing 

77 

.5% 

14 

.5 

22 

Compared  with  Alberta  Program  of  Studies 

76 

.1% 

16 

26 

Solicited  teacher  analysis  of  data 

70 

.4% 

17 

.5 

28 

Encouraged  consideration  of  alternatives 

70  .4% 

17 

.5 

37 

Appeal  procedure  available 

69 

.0% 

19 

11 

Recorded  data  on  student  responses 

60 

.6% 

20 

9 

Made  notes  of  classroom  dialogue 

59 

.2% 

21 

.5 

10 

Recorded  data  on  teacher  questions 

59 

.2% 

21 

.5 

13 

Recorded  data  on  patterns  of  response 

56 

.3% 

23 

2 

Standards  of  performance  developed 

54 

.9% 

24 

3 

Met  prior  to  observation 

49 

.3% 

25 

18 

Recorded  data  on  specific  student 

47 

.9% 

26 

33 

Developed  recommendations  with  teacher 

45 

.1% 

27 

15 

Recorded  data  on  teacher  movement 

43 

.7% 

28 

16 

Recorded  data  on  student  movement 

40 

.8% 

29 

1 

Had  written  job  description 

35 

.2% 

30 

.5 

31 

Recommended  resources 

35 

.2% 

30 

.5 

34 

Developed  a  plan  of  assistance 

32.4% 

32 

6 

Suggested  observational  techniques 

29 

.6% 

33 

29 

Listened  more  than  I  talked 

28 

.2% 

34 

7 

Teacher  chose  observational  strategies 

22 

.5% 

35 

14 

Made  audio  recordings 

1 

.4% 

36 

17 

Made  video  recordings 

0% 

37 
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TABLE  19.     Percentage  of  Other  Central  Office  Staff  Who 
Responded   "Yes"   to  Items  on  the  Modified 
Acheson/Shinn  Instrument 


Percentage 

of  "Yes" 
Responses 


Item 

#  Descriptor 

n  =  65 

Rank 

20 

Observed  for  at  least  30  minutes 

100  % 

1 

.5 

25 

Met  with  teacher  after  observation 

100  % 

1 

.5 

21 

Examined  teacher's  planbook 

95  .4% 

4 

.5 

24 

Examined  student  work 

95  .4% 

4 

.5 

8 

Recorded  impressions  about  classroom 

95  .4% 

4 

.5 

35 

Teacher  received  a  copy  of  report 

95  .4% 

4 

.5 

12 

Recorded  data  on  on-task  behavior 

93  .8% 

8 

30 

Gave  praise  and  encouragement 

92  .3% 

7 

27 

Encouraged  teacher  inferences  &  opinions 

90  .8% 

9 

5 

Found  teacher  concerns 

89  .2% 

10 

.5 

19 

Recorded  student/teacher  interactions 

89.2% 

10 

.5 

37 

Appeal  procedure  available 

87  .7% 

12 

.5 

32 

Developed  recommendations 

87  .7% 

12 

.5 

36 

Teacher  could  respond  in  writing 

86  .2% 

14 

4 

Found  teacher  expectations  of  students 

84  .6% 

15 

26 

Solicited  teacher  analysis  of  data 

83.1% 

16 

28 

Encouraged  consideration  of  alternatives 

81  . 5  % 

17 

.5 

10 

Recorded  data  on  teacher  questions 

81  .5% 

17 

.5 

11 

Recorded  data  on  student  responses 

80  .0% 

19 

3 

Met  prior  to  observation 

76  .9% 

20 

.  5 

23 

Examined  record  of  student  achievement 

76  .9% 

20 

.5 

22 

Compared  with  Alberta  Program  of  Studies 

67.7% 

22 

9 

Made  notes  of  classroom  dialogue 

66  .2% 

23 

18 

Recorded  data  on  specific  student 

64  .6% 

24 

33 

Developed  recommendations  with  teacher 

63.1% 

25 

2 

Standards  of  performance  developed 

58  .5% 

26 

31 

Recommended  resources 

53  .8% 

27 

15 

Recorded  data  on  teacher  movement 

52  .3% 

28 

16 

Recorded  data  on  student  movement 

50  .8% 

29 

1 

Had  written  job  description 

49  .2% 

30 

13 

Recorded  data  on  patterns  of  response 

47.7% 

31 

34 

Developed  a  plan  of  assistance 

44  .6% 

32 

29 

Listened  more  than  I  talked 

35  .4% 

33 

7 

Teacher  chose  observational  strategies 

29.2% 

34 

6 

Suggested  observational  techniques 

27  .7% 

35 

14 

Made  audio  recordings 

1.5% 

36 

.5 

17 

Made  video  recordings 

1  5% 

36 

.5 
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TABLE  20.     Percentage  of   Principals  Who  Responded   "Yes"  to 
Items  on  the  Modified  Acheson/Shinn  Instrument 


Percentage 

of  "Yes" 
Responses 


Item 

#                        Descriptor  i 

a  =  211 

Rank 

25 

Met  with  teacher  after  observation 

97  .6% 

1 

35 

Teacher  received  a  copy  of  report 

95.7% 

2 

30 

Gave  praise  and  encouragement 

94  .3% 

3 

20 

Observed  for  at  least  30  minutes 

93  .8% 

4 

37 

Appeal  procedure  available 

91  .5% 

5 

27 

Encouraged  teacher  inferences  &  opinions 

89  .1% 

6 

36 

Teacher  could  respond  in  writing 

88  .6% 

7 

19 

Recorded  student/teacher  interactions 

87  .7% 

8 

12 

Recorded  data  on  on-task  behavior 

86  .7% 

9 

.5 

3 

Met  prior  to  observation 

86.7% 

9 

.5 

24 

Examined  student  work 

86  .3% 

11 

8 

Recorded  impressions  about  classroom 

85.3% 

12 

4 

Found  teacher  expectations  of  students 

82.5% 

13 

.5 

5 

Found  teacher  concerns 

82.5% 

13 

.5 

26 

Solicited  teacher  analysis  of  data 

80  .6% 

15 

21 

Examined  teacher's  planbook 

74  .4% 

16 

28 

Encouraged  consideration  of  alternatives 

67  .3% 

17 

11 

Recorded  data  on  student  responses 

65.9% 

18 

.5 

32 

Developed  recommendations 

65.9% 

18 

.5 

33 

Developed  recommendations  with  teacher 

64  .9% 

20 

10 

Recorded  data  on  teacher  questions 

63  .0% 

21 

22 

Compared  with  Alberta  Program  of  Studies 

62  .1% 

22 

2 

Standards  of  performance  developed 

60  .7% 

23 

9 

Made  notes  of  classroom  dialogue 

53  .1% 

24 

13 

Recorded  data  on  patterns  of  response 

51  .2% 

25 

23 

Examined  record  of  student  achievement 

48.8% 

26 

18 

Recorded  data  on  specific  student 

47  .4% 

27 

15 

Recorded  data  on  teacher  movement 

45.5% 

28 

.5 

16 

Recorded  data  on  student  movement 

45.5% 

28 

.5 

29 

Listened  more  than  I  talked 

43.6% 

30 

1 

Had  written  job  description 

43.1% 

31 

7 

Teacher  chose  observational  strategies 

41.2% 

32 

31 

Recommended  resources 

37  .9% 

33 

6 

Suggested  observational  techniques 

37  .0% 

34 

34 

Developed  a  plan  of  assistance 

34  .6% 

35 

14 

Made  audio  recordings 

1  .9% 

36 

17 

Made  video  recordings 

0% 

37 
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TABLE  21.     Percentage  of  Evaluators  Responding   "Yes"  to 
Items  on  the  Modified  Acheson/Shinn  Instrument 


Percentage 

of  "Yes" 
Responses 


I  tern 

#                        Descriptor  r 

1  =  347 

Rank 

25 

Met  with  teacher  after  observation 

98  .3% 

1 

20 

Observed  for  at  least  30  minutes 

95.4% 

2 

35 

Teacher  received  a  copy  of  report 

95  .1% 

3 

30 

Gave  praise  and  encouragement 

91  .9% 

4 

19 

Recorded  student/teacher  interactions 

89  .3% 

5  .5 

24 

Examined  student  work 

89  .3% 

5  .5 

8 

Recorded  impressions  about  classroom 

88  .8% 

7 

12 

Recorded  data  on  on-task  behavior 

87.0% 

8.5 

27 

Encouraged  teacher  inferences  &  opinions 

87  .0% 

8  .5 

37 

Appeal  procedure  available 

86.2% 

10 

36 

Teacher  could  respond  in  writing 

85.9% 

11 

5 

Found  teacher  concerns 

85  .3% 

12 

4 

Found  teacher  expectations  of  students 

82.1% 

13 

21 

Examined  teacher's  planbook 

81  .8% 

14 

26 

Solicited  teacher  analysis  of  data 

79  .0% 

15 

3 

Met  prior  to  observation 

77  .2% 

16 

32 

Developed  recommendations 

73.8% 

17 

28 

Encouraged  consideration  of  alternatives 

70  .6% 

18 

1 1 

Recorded  data  on  student  responses 

67  .4% 

19 

22 

Compared  with  Alberta  Program  of  Studies 

66  .0% 

20 

10 

Recorded  data  on  teacher  questions 

65.7% 

21 

23 

Examined  record  of  student  achievement 

61.4% 

22 

33 

Developed  recommendations  with  teacher 

60  .5% 

23 

2 

Standards  of  performance  developed 

59  .1% 

24 

9 

Made  notes  of  classroom  dialogue 

56  .8% 

25 

13 

Recorded  data  on  patterns  of  response 

51  .6% 

26 

18 

Recorded  data  on  specific  student 

50  .7% 

27 

15 

Recorded  data  on  teacher  movement 

46  .4% 

28 

16 

Recorded  data  on  student  movement 

45.5% 

29 

1 

Had  written  job  description 

42.7% 

30 

31 

Recommended  resources 

40  .3% 

31 

29 

Listened  more  than  I  talked 

38  .9% 

32 

34 

Developed  a  plan  of  assistance 

36  .0% 

33 

7 

Teacher  chose  observational  strategies 

35.2% 

34 

6 

Suggested  observational  techniques 

33  .7% 

35 

14 

Made  audio  recordings 

1.7% 

36 

17 

Made  video  recordings 

0.3% 

37 
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TABLE  22.     Comparative   Ranking  of   "Yes"   Responses  to  Items 
on  the  Acheson/Shinn  Instrument 


Rank 


Item 

#  Descriptor 

A 

S 

0 

P 

25 

Met  with  teacher  after  observation 

1 

1 

1 

.5 

1 

20 

Observed  for  at  least  30  minutes 

2 

2 

1 

.5 

4 

35 

Teacher  received  a  copy  of  report 

3 

5 

4 

.5 

2 

30 

Gave  praise  and  encouragement 

4 

10 

7 

3 

19 

Recorded  student/teacher  interactions 

5 

.5 

3 

10 

.  5 

8 

24 

Examined  student  work 

5 

.5 

5 

4 

.  5 

11 

8 

Recorded  impressions  about  classroom 

7 

5 

4 

.5 

12 

12 

Recorded  data  on  on-task  behavior 

8 

.5 

12 

8 

9 

.5 

27 

Bnooiiraged  teacher  inferences  &  opinions 

8 

.5 

14 

.5 

9 

6 

37 

Appeal  procedure  available 

10 

19 

12 

.5 

5 

36 

Teacher  could  respond  in  writing 

11 

14 

.5 

14 

7 

5 

Found  teacher  concerns 

12 

8 

10 

.5 

13 

.5 

4 

Found  teacher  expectations  of  students 

1  3 

1  3 

1  5 

1 3 

.  5 

21 

Examined  teacher's  planbook 

14 

7 

4 

.5 

16 

26 

Solicited  teacher  analysis  of  data 

15 

17 

.5 

16 

15 

3 

Met  prior  to  observation 

16 

25 

20 

.5 

9 

.5 

3  2 

Developed  recommendations 

1 7 

10 

1 2 

.  5 

1  8 

.  5 

28 

Encouraged  consideration  of  alternatives 

18 

17 

.5 

17 

.5 

17 

11 

Recorded  data  on  student  responses 

19 

20 

19 

18 

.5 

22 

Compared  with  Alberta   Program  of  Studies 

20 

16 

22 

22 

10 

Recorded  data  on  teacher  questions 

21 

21 

=  5 

17 

.5 

21 

23 

Examined  record  of  student  achievement 

22 

10 

20 

.5 

26 

33 

Developed  recommendations  with  teacher 

23 

27 

25 

20 

2 

Standards  of  performance  developed 

24 

24 

26 

23 

9 

Made  notes  of  classroom  dialogue 

25 

21 

.5 

23 

24 

13 

Recorded  data  on  patterns  of  response 

26 

33 

21 

35 

18 

Recorded  data  on  specific  student 

27 

26 

24 

27 

15 

Recorded  data  on  teacher  movement 

28 

28 

28 

28 

.5 

16 

Recorded  data  on  student  movement 

29 

29 

29 

28 

.5 

1 

Had  written  job  description 

30 

30 

.5 

30 

31 

31 

Recommended  resources 

31 

30 

.5 

27 

33 

29 

Listened  more  than  I  talked 

32 

34 

33 

30 

34 

Developed  a  plan  of  assistance 

33 

32 

32 

35 

7 

Teacher  chose  observational  strategies 

34 

35 

34 

32 

6 

Suggested  observational  techniques 

35 

33 

35 

34 

14 

Made  audio  recordings 

36 

36 

36 

.5 

36 

17 

Made  video  recordings 

37 

37 

36 

.5 

37 

Spearman  Rank  Order  Correlation  Coefficients: 
S:0  r  =   .94;   p  <  .01 

S:P  r  =    .84;   p  <  .01 

0:P  r=.90;p<.01 


*  A  =  All  respondents;   S  =  Superintendents;  O  =  Other  Central  Office 
Staff;   P  =  Principals 
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24),  and  the  examination  of  the  teacher's  planbook  (Item 
21).     In  addition,   superintendents  placed  a  much  greater 
emphasis  upon  the  examining  the  record  of  student 
achievement  (Item  23). 

Principals,  more  than  other  central  office  staff,  and 
superintendents,  encouraged  teacher  input  in  the  post 
observation  conference   (Item  27),  gave  praise  and 
encouragement  more  frequently   (Item.  30),  and  provided  more 
opportunity  for  the  teacher  to  respond  to  the  report  in 
writing   ( Item  36). 

Superintendents  developed  recommendations  to  help  the 
teacher  improve  instruction  (Item  32)  more  frequently  than 
other  central  office  staff  or  principals. 

Table  23  reports  the  percentage  of  superintendents, 
other  central  office  staff,  and  principals  who  attained  the 
minimum  criteria  in  the  areas  of  due  process,  the 
improvement  of  instruction,  and  teacher  perception.  The 
minimum  criteria  for  satisfactory  evaluation  in  each  of 
these  areas  was  defined  as  the  subset  of  the  items  receiving 
a  mean  score  of  between  one  and  1.75   (as  rated  by  the 
experts)  on  the  modified  Acheson/Shinn  Instrument. 

Of  the  347  evaluations  reported,  only  15  met  the 
minimum  criteria  for  assuring  due  process.     Other  central 
office  staff  achieved  a  higher  rate  of  meeting  the  criteria 
than  did  superintendents,  who  in  turn  did  better  than 
principals.     An  analysis  of  variance  of  the  mean  scores  on 
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TABLE  23.     Percentage  of  the  Study  Group  Attaining  the 
Minimum  Criteria  in  the  Areas  of  Due  Process,  the 
Improvement  of  Instruction  and 
Teacher  Perception 


Superintendents 
N  =  71 

Other  Central 
Office  Staff 
N  =  65 

Pr inc  ipals 
N  =  211 

Due  Process 

4.2% 

10  .8% 

2.4% 

Improvement 
of 

Instruction 

1  .4% 

0% 

1.4% 

Teacher 
Perception 

2.8% 

16.9% 

8.1% 

93 


due  process  items  showed  a  difference  in  the  means, 
significant  at  the    .01   level.     Other  central  office  staff 
scored  higher  than  principals  who  in  turn  were  higher  than 
superintendents.     The  mean  score  for  all  respondents  was 
12.5   (out  of  17  due  process  items). 

The  items  rated  essential  for  assuring  due  process  that 
were  not  used  by  more  than  50%  of  the  respondents  included: 

Item  34,   the  development  of  a  plan  of  assistance 
(superintendents,  other  central  office  staff,  and 
pr inc  ipals ) . 

Item  31,   the  recommendation  of  resources 
(superintendents  and  principals). 

Item  23,   the  examination  of  the  record  of  student 
achievement  (principals). 

Item  33,   the  development  of  recommendations  in 
cooperation  with  the  teacher  (superintendents). 

One  superintendent  and  three  principals  met  the  minimum 
criteria  for  the  improvement  of  instruction.     Analysis  of 
variance  of  the  mean  scores  on  improvement  of  instruction 
items  showed  a  difference  in  the  means,   significant  at  the 
.01  level,  with  other  central  office  staff  higher  than 
principals  who  in  turn  were  higher  than  superintendents. 
The  mean  score  on  the  25  improvement  of  instruction  items 
was  1 8  . 6  . 

The  items  essential  for  improvement  of  instruction  that 
were  not  used  by  more  than  50%  of  the  respondents  included: 
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Item  34,   the  development  of  a  plan  of  assistance 
(superintendents,  other  central  office  staff,  and 
pr inc  ipals ) . 

Item  29,   I  listened  more  than   I  talked 
(superintendents,  other  central  office  staff,  and 
principals ) . 

Item  31,   the  recommendation  of  resources 
(superintendents  and  principals). 

Item  33,   the  development  of  recommendations  in 
cooperation  with  the  teacher  (superintendents). 

Item  3,  meeting  prior  to  the  observation 
(superintendents). 

Twenty  nine  respondents,   two  superintendents   (2.8%),  11 
other  central  office  staff   (16.9%),   and  17  principals  (8.1%) 
met  the  minimum  criteria  established  by  teachers.  Analysis 
of  variance  of  the  mean  scores  on  teacher  perception  items 
showed  a  difference  in  the  means,   significant  at  the  .01 
level,  with  other  central  office  staff  higher  than 
principals  who  in  turn  were  higher  than  superintendents.  Of 
the  16  items  identified  by  teachers  as  essential  to  formal 
teacher  evaluation,   respondents  achieved  a  mean  score  of 
12.5. 

The  items  perceived  by  teachers  to  be  essential   in  a 
formal  evaluation  that  were  not  used  by  more  than  50%  of  the 
respondents  included: 

Item  34,   the  development  of  a  plan  of  assistance 


(superintendents,  other  central  office  staff,  and 
principals )  . 

Item  31,   the  recommendation  of  resources 
(superintendents  and  principals). 

Item  33,   the  development  of  recommendations  in 
cooperation  with  the  teacher  (superintendents). 

Item  3,  meeting  prior  to  the  observation 
(superintendents). 

Relationships  Between  Demographic  Factors  and  Scores 
on  the  Modified  Acheson/Shinn  Instrument" 

Introduction 

Of  the  37  items  on  the  Modified  Acheson/Shinn 
Instrument,   26  were  rated  by  the  experts  as  essential  to  the 
improvement  of  instruction  and  the  assurance  of  due  process 
in  formal  teacher  evaluation.     Of  the  remaining  11  items, 
all  but  items  14  and  17   (audio  and  video  recording  of 
lessons)  were  rated  by  all  groups  as  desirable  or  acceptable 
evaluation  strategies.     Experts  in  due  process  rated  items 
14  and  17  as  marginally  unacceptable;  however,   the  use  of 
audio  and  video  tapes  in  the  evaluation  of  teaching   is  a 
well  established  practice  and  is  one  of  the  recommended 
strategies  in  the  clinical  supervision  model   (Acheson  & 
Gall,   1980,   p.   132) . 

The  scores  on  the  entire  instrument  correlate  highly 
with  the  scores  on  the  subsets  of  items  rated  as  essential 
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to  assuring  due  process   (r  =   .89)   and  for  the  improvenient  of 
instruction   (r  =    .95);   thus,   the  score  on  the   instrument  is 
regarded  as  a  determinant  of  the   "completeness"  of  the 
evaluation.     The  higher  the  score,   the  more  complete  the 
evaluation. 

The  section  that  follows  reports  the  statistical 
relationships  between  the  respondents  '  demographic 
characteristics  and  the  completeness  of  their  last  formal 
teacher  evaluation.     In  the  interests  of  readability, 
statistical  data  will  be  reported  in  Appendix  D,  and  the 
text  on  the  following  pages  will  merely  report  and  discuss 
the  statistical  patterns. 

Experience 

To  explore  the  relationship  between  years  of  experience 
as  a  teacher  evaluator  and  the  completeness  of  the 
evaluation,   the  Pearson  product-moment  correlation 
coefficient  was  calculated.     Initial  calculation  of  the 
coefficient  suggested  a  very  slight,  non  significant, 
negative  correlation  between  the  number  of  years  of 
experience  as  a  teacher  evaluator  and  the  score  on  the 
instrument.     A  recalculation  of  the  coefficient  broken  down 
by  administrator  category,   revealed  no  statistically 
significant  relationship  for  other  central  office  staff  and 
principals.     However,   superintendents  showed  a  negative 
correlation   (r  =  -0.24)   significant  at  the    .05  level.  This 
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means  that  more  experienced  superintendents  in  this  study 
tended  to  score  lower  than  those  with  less  experience. 

It  is  possible  that  this  somewhat  unexpected  finding  is 
caused  by  the  fact  that  almost  one  third  of  Alberta 
superintendents  have  16  or  more  years  of  experience  as  a 
teacher  evaluator,  and  thus  may  well  have  learned  their 
evaluation  strategies  in  a  period  when  the  inspectoral  model 
was  prevalent,   strategies  that  are  incompatible  with  the 
clinical  supervision  model  upon  which  the  instrument  is 
based  . 

University  Courses 

Calculation  of  the  Pearson  correlation  coefficient 
shows  a  significant  relationship  (r  =   .14;  p  <   .01)  between 
the  score  on  the  instrument  and  the  number  of  university 
courses  in  the  evaluation  of  teachers  undertaken  by  the 
respondents.     The  more  university  courses  taken,   the  higher 
the  score  on  the  instrument.     Superintendents  averaged  the 
greatest  number  of  university  courses,   followed  by  other 
central  office  staff,  and  principals   (Table  2). 

Inservice  Training 

No  statistically  significant  relationship  was  found 
between  the  score  on  the  instrument  and  inservice  training; 
however,  a  low  but  significant   (r  =   .1;  p  <   .05)  positive 
correlation  existed  between  inservice  training  and  the  score 
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on  the  due  process  subset.     Respondents  with  more  inservice 
training  scored  higher  on  the  due  process  subset.  Other 
central  office  staff  had  more  inservice  training  than 
superintendents  or  principals   (Table  3). 

Number  of  Teachers 

To  examine  the  relationship  between  the  completeness  of 
the  evaluation  and  the  number  of  teachers  supervised  (for 
superintendents  this  means  the  number  of  teachers  in  the 
jurisdiction;   for  other  central  office  staff,   the  number  of 
teachers  in  their  area  of  responsibility;  and  for 
principals,   the  number  of  teachers  in  their  school),  the 
Pearson  product-moment  correlation  coefficient  was 
calculated.     A  statistically  significant   (r  =   .15;  p  <  .01), 
positive  relationship  was  found  between  the  score  on  the  due 
process  subset  and  the  number  of  teachers  supervised.  No 
significant  relationships  were  found  between  the  number  of 
teachers  supervised  and  the  completeness  of  the  evaluation, 
the  improvement  of  instruction  subset,  or  the  teacher 
preference  subset. 

When  the  same  test  was  used  to  examine  the  relationship 
between  the  number  of  teachers  evaluated  and  the 
completeness  of  the  evaluation,  a  positive  correlation, 
significant  at  the    .05  level,  was  found.     In  addition, 
significant  correlations  exist  between  the  number  of 
teachers  evaluated  and  the  due  process  subset   (r  =  .16; 
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p  <    .01),   and  the  improvement  of  instruction  subset   (r  =  .1? 
p  <    .05).     No  statistically  significant  relationship  was 
found  in  the  case  of  the  teacher  preference  subset. 

Category  of  Teacher 

To  determine  what  differences  existed  in  the  evaluative 
practices  of  respondents  with  respect  to  the  category  of 
teacher  evaluated   (i.e.,   first  year,  experienced  but  new  to 
the  school/system,  or  tenured),   an  analysis  of  variance  of 
the  mean  scores  of  respondents,   broken  down  by  teacher 
category,  was  carried  out.     No  significant  differences  in 
the  means  were  found. 

Presence  of  Policy 

Respondents  were  asked  to  indicate   (yes,  no,  or  unsure) 
whether  their  jurisdiction  had  a  written  policy  on  teacher 
evaluation   (Table  4).     Although  the  mean  for  "yes"  responses 
was  higher  than  that  for  the  "unsure"  responses,  which  in 
turn  was  higher  than  for  the   "no"  responses,   an  analysis  of 
variance  showed  no  significant  differences  between  the  means 
indicating  that  chance  factors  alone  could  account  for  the 
var iat  ion  . 

This  means  that  the  perceived  presence   (or  absence)  of 
policy  has  no  relationship  with  the  completeness  of  the 
evaluation  carried  out  by  respondents. 
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Adherence  to  Policy 

No  significant  relationship  was  found  between  the 
reported  level  of  adherence  to  the  teacher  evaluation  policy 
and  the  completeness  of  the  evaluation,   nor  to  the  due 
process,   improvement  of  instruction,  or  teacher  preference 
subsets.     This  means  that  the  completeness  of  the  evaluation 
was  unrelated  to  the  degree  of  adherence  to  teacher 
evaluation  policy. 

Classroom  Visits 

The  Pearson  product-moment  correlation  coefficient  was 
calculated  to  examine  the  relationship  between  the  number  of 
classroom  visits  made,  and  the  completeness  of  the  evaluation. 
A  statistically  significant  positive  correlation  was  found 
(r  =   .13;  p  -   .01).     Similarly,   the  relationship  between  the 
time  spent  in  classroom  observation,   and  the  completeness  of 
the  evaluation  was  examined,  and  a  significant  positive 
correlation  was  found   (r  =   .3;  p  <  .01). 

The  possible  relationships  between  grade  levels 
observed,  or  subjects  observed,  and  the  completeness  of  the 
evaluation  were  examined  by  using  an  analysis  of  variance  of 
the  mean  scores  of  respondents,   broken  down  by  grade  or 
subject  level.     No  significant  differences  in  the  means  were 
found  relating  to  the  grade  level  observed.     A  similar 
result  was  obtained  for  all  subjects  except  for  science. 
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Respondents  who  observed  science  lessons  scored 
significantly  higher   (p  <    .05)   than  those  who  did  not. 

This  means  that  evaluators  who  make  more  classroom 
visits,  or  who  spend  more  time  in  observations,   tend  to  do  a 
more  complete  evaluation  than  those  who  visit  less  often,  or 
spend  less  time  in  observation.     The  grade  level  and  subject 
observed   (except  for  science)  make  no  difference  to  the 
completeness  of  the  evaluation. 

Purpose  of  the  Evaluation 

Respondents  were  given  a  list  of  ten  purposes  for 
evaluation  and  were  asked  to  indicate  the  primary  purpose  of 
the  last  formal  evaluation.     The  difference  between  mean 
scores  on  the  modified  Acheson/Shinn  instrument  (the 
completeness  measure)  of  respondents  who  checked  a  given 
item,  and  those  who  did  not,  was  examined  using  analysis  of 
variance.     No  significant  differences  in  the  means  were 
found  when  the  purpose  was:     awarding  a  permanent  contract, 
providing  a  basis  for  dismissal,  providing  a  basis  for 
transfer,  providing  a  basis  for  promotion,  providing  a  basis 
for  retention  in  position,  determining  the  inservice  needs 
of  the  teacher,  or  when  the  respondent  checked  the  "other" 
category . 

A  significantly  higher  mean  score  was  obtained  by 
respondents  when  the  purpose  was  to  record  the  performance 
of  a  resigning  or  retiring  teacher   (p  <    .05)  or  when  the 
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purpose  was  to  improve  the  performance  of  the  teacher   (p  < 
.01)  . 

A  significantly  lower  mean  score  was  obtained  when  the 
purpose  was  related  to  recommending  permanent  certification. 

This  means  that  respondents  whose  primary  purpose  was 
to  improve  the  performance  of  the  teacher,  or  to  record  the 
performance  of  a  resigning  or  retiring  teacher,   tended  to  do 
a  more  complete  evaluation  than  respondents  who  did  not  have 
these  primary  purposes.     Respondents  whose  primary  purpose 
was  related  to  recommending  for  permanent  certification 
tended  to  do  a  less  complete  evaluation. 

Initiating  Factors 

Respondents  were  given  a  list  of  six  initiating  factors 
and  were  asked  to  indicate  the  primary  initiating  factor  in 
the  last  formal  evaluation.     The  difference  between  mean 
scores  on  the  modified  Acheson/Shinn  instrument  (the 
completeness  measure)  or  respondents  who  checked  a  given 
item,   and  those  who  did  not,  was  examined  using  analysis  of 
variance.     No  significant  differences  in  the  means  were 
found  when  the  initiating  factor  was:     a  parental  complaint, 
concerns  arising  from  informal  observations,  a  teacher 
request,   the  need  for  documentation,  or  when  respondents 
checked  the   "other"  category.     The  mean  score  was 
significantly  higher   (p  <    .05)  when  the  evaluation  was 
initiated  as  a  routine  evaluation  according  to  district 
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policy  than  when  it  was  not. 

Outcome  of  the  Evaluation 

Respondents  were  given  a  list  of  nine  possible  outcomes 
and  were  asked  to  indicate  the  outcome  of  the  last  formal 
evaluation.     The  difference  between  mean  scores  on  the 
modified  Acheson/Shinn  instrument   (the  completeness  measure) 
of  respondents  who  checked  a  given  item,  and  those  who  did 
not,  was  examined  using  analysis  of  variance.  No 
significant  differences  in  the  means  were  found  when  the 
outcome  was:     the  teacher  was/will  be  awarded  a  permanent 
contract,   the  teacher  was/will  be  dismissed,   the  teacher 
was/will  be  promoted,   the  teacher  will  continue  in  present 
position,   the  teacher  was/will  be  evaluated  by  another 
evaluator,   the  teacher  challenged  the  evaluation  report,  or 
when  the  respondents  checked  the   "other"  category. 

The  mean  score  for  respondents  was  significantly  higher 
(p  <    .01)  when,   as  a  result  of  the  evaluation,   an  inservice 
plan  was  developed  for  the  teacher,  and  significantly  lower 
(p  <    .01)  when  the  evaluation  resulted  in  a  recommendation 
for  permanent  certification. 
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CHAPTER  V 

SUMMARY,    CONCLUSIONS,    AND  RECOMMENDATIONS 
Introduction 

The  case  of  James  Keegstra   (Keegstra  v.  County  of 
Lacombe ,   1983),  who  was  dismissed  from  his  teaching  position 
after  it  was  revealed  that  he  had  been  teaching  a  social 
studies  program  that  differed  substantially  from  the 
authorized  Alberta  curriculum,   rekindled  public  and 
political   interest  in  the  evaluation  of  teachers.  This 
renewed  interest  contributed  to  the  development,   in  1984,  of 
a  teacher  evaluation  policy  by  Alberta  Education.  In 
essence,   this  policy  requires  that  Alberta  school 
jurisdictions  put  into  place  fair  and  consistent  board 
policies  to  assure  the  regular  evaluation  of  teachers  for 
the  purposes  of  (1)  assisting  in  the  professional 
development  of  teachers,  and   (2)   taking  appropriate  action 
with  respect  to  teachers  whose  performance  is  unacceptable. 

This  study  provides  benchmark  data  for  assessing  the 
impact  of  the  Alberta  Education  policy,  and  information  to 
stakeholder  groups  on  current  practices  and  the  minimum 
criteria  to  be  met  for  satisfactory  teacher  evaluation. 

The  primary  purpose  of  this  study  was  to  describe  the 
state  of  formal  teacher  evaluation  in  Alberta  in  the  1983-84 
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school  year.     The  specific  aims  were: 

1.  To  collect  demographic   information  about  the 
evaluators  and  teachers  involved  in  formal  teacher 
evaluation  in  Alberta. 

2.  To  describe  the  initiating  factors,  intended 
purposes  and  outcomes  of  formal  teacher  evaluation. 

3.  To  describe  the  procedures  used  by  evaluators  in 
formally  evaluating  teachers. 

4.  To  analyze  the  quality  of  teacher  evaluation 
procedures  with  respect  to  minimum  requirements  to 
improve  instruction  and  to  assure  due  process. 

5.  To  identify  the  relationships  between  selected 
demographic  factors  and  the  completeness  of  the 
evaluation . 

The  Literature 

The  literature  related  to  teacher  evaluation  is 
extensive,  diverse,  and  sometimes  contradictory.     The  review 
of  the  literature  for  this  study  traces  the  development  of 
evaluation  from  its  beginnings  as  an  inspectoral,  autocratic 
process,   through  the  "scientific"  and  "democratic"  models  of 
the  early  twentieth  century  to  the  multigoal,  democratic 
interactive  process  exemplified  by  Acheson's   (1982)  model  of 
the  clinical  supervision  process. 

The  literature  identifies  a  great  many  purposes  for 
teacher  evaluation,  but  they  can  all  be  incorporated  into 
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two  major  categories,   namely  purposes  related  to  the 
improvement  of  the  teacher's   instructional  performance,  and 
purposes  involving  judgemental   functions  related  to  decision 
making  and  teacher  competency. 

The  profound  impact  of  the  evaluation  of  teachers  on 
student  learning  is  dealt  with  in  some  early  literature,  but 
more  recent  research  has  focussed  upon  teaching  behaviors 
rather  than  student  outcomes. 

Teacher  evaluation  in  Alberta  is  the  subject  of  a  great 
many  opinion  articles  but  few  research  studies.     The  studies 
that  have  been  made  tend  to  focus  upon  the  superintendent's 
perception  of  the  policies  and  practices  prevalent  in  their 
jurisdiction;  however,  one  recent  study   (Townsend,  1984) 
documented  the  process  of  implementation  of  a  new  policy  on 
teacher  supervision  and  evaluation,  and  demonstrated  that 
high  levels  of  acceptance  and  participation,  and 
satisfaction  with  the  process  can  be  attained  through 
appropriate  implementation  strategies. 

The  Design  of  the  Study 

To  assess  the  state  of  formal  teacher  evaluation  in 
Alberta,   the  following  steps  were  taken: 

1.     A  three  part  questionnaire,   based  upon  existing 
instruments,  was  developed.     Section  one  collected 
demographic  data  about  the  responding  evaluator.  Section 
two  addressed  the   "last  formal  teacher  evaluation  carried 
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out,"  and  collected  data  related  to  the  initiating  factors, 
purposes,   and  outcomes  of  that  evaluation.     Section  three, 
the  Modified  Acheson/Shinn  instrument,  collected  data  on  the 
supervisory  practices  used   in  the  last  formal  teacher 
evaluation . 

2.  The  questionnaire  was  administered  to  a  study  group 
consisting  of  all  Alberta  superintendents,  all  other  central 
office  staff  who  were  members  of  the  Conference  of  Alberta 
School  Superintendents  as  of  December  1983,   and  a  25%  sample 
of  Alberta  school  principals. 

3.  Minimum  criteria  for  satisfactory  evaluation  in  the 
areas  of  "due  process"  and  "improvement  of  instruction"  were 
established  by  submitting  Section  3  of  the  questionnaire, 
the  Modified  Acheson/Shinn  instrument,   to  experts  for 
adjudication.     Teacher  perceptions  of  appropriate  evaluation 
strategies  were  assessed  by  administering  the  Modified 
Acheson/Shinn  instrument  to  a  sample  of  teachers. 

4.  Data  from  the  questionnaires  were  processed  using 
the  SPSSX  Sprint  package.     Frequency  distributions  and  means 
were  generated  for  all  variables.     Ratings  by  experts  and 
teachers  of  the  37   items  on  the  Modified  Acheson/Shinn 
instrument  were  summed  and  averaged  and  three  subsets 
delineating  the  minimum  criteria  for  satisfactory  evaluation 
in  the  areas  of  due  process,   improvement  of  instruction,  and 
teacher  perception  were  generated.     Demographic  factors  were 
described  and  the  relationships  between  the  demographic 
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variables  and   the  scores  on  the  Modified  Acheson/Shinn 
instrument  and   its  subsets  were  examined. 

Findings 

1.  On  average,   superintendents  had  13.8  years 
experience   in  teacher  evaluation,  had  undertaken  1.87 
university  courses  related  to  teacher  evaluation,  and  had 
engaged  in  18.5  hours  of  related  inservice  training   in  the 
last  five  years.     By  comparison,  other  central  office  staff 
had  11.4  years  experience,   1.76  university  courses,   and  23.4 
hours  of  inservice  training.     Principals  averaged  9.3  years 
experience,  had  taken  1.0  university  courses,   and  14.9  hours 
of  inservice  training. 

2.  Not  all  respondents  were  involved  in  teacher 
evaluation  during  the  study  period  but  71  .7%  of 
superintendents,   67%  of  other  central  office  staff,  and 
73.3%  of  principals  reported  that  they  had  evaluated  one  or 
more  teachers.     Of  the  respondents  who  did  evaluate 
teachers,  superintendents  averaged  20.8  teachers,  other 
central  office  staff,   25.1   teachers  and  principals,  8.7 
teachers . 

3.  During  the  study  period,   respondents  reported  a 
total  of  4,924  evaluations  performed.     Of  these, 
superintendents  did  29.5%,  other  central  office  staff  did 
33.1%,   and  principals  did  37.3%.     Of  the  evaluations 
performed,   21.9%  were  of  first  year  teachers,   17.3%  were  of 
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experienced  teachers  new  to  the  school/system,   and  60.8% 
were  tenured. 

4.  Respondents,  especially  superintendents,  generally 
rated  teacher  evaluation  as  a  high  priority  item  in  relation 
to  all  their  other  duties,   but  most  respondents  considered 
the  time  available  to  be  inadequate.     Respondents  who  rated 
teacher  evaluation  as  a  high  priority  scored  significantly 
higher  on  the  Modified  Acheson/Shinn  instrument,   the  measure 
of  the  completeness  of  the  evaluation;  however,  the 
perceived  amount  of  time  available  had  no  significant  impact 
on  the  completeness  of  the  evaluation. 

5.  Over  80%  of  respondents  indicated  that  their 
jurisdiction  had  a  written  policy  on  formal  teacher 
evaluation,   but  less  than  50%  of  the  respondents  who 
evaluated  teachers  followed  the  policy  "strictly  to  the 
letter."     Neither  the  presence  of  policy,  nor  the  degree  to 
which  it  was  followed,   related  significantly  to  the 
completeness  of  the  evaluation. 

6.  Over  80%  of  respondents  made  two  or  more  classroom 
visits  during  the  last  formal  teacher  evaluation,   and  the 
average  time  spent  in  classroom  observation  was  over  two 
hours.     Junior  high  teachers  comprised  the  largest  group  of 
teachers  evaluated  and  language  arts  and  math  were  the 
subjects  most  commonly  observed.     Respondents  who  made  more 
classroom  visits  or  spent  more  time  in  classroom  observation 
during  the  last  formal  teacher  evaluation  scored 
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significantly  higher  on  the  completeness  measure  than  those 
who  visited  less  often,  or  spent  less  time  in  observation. 
The  grade  level  and  subject  observed   (except  science)  was 
not  significantly  related  to  the  completeness  of  the 
evaluation . 

7.  The  majority  of  the  last  formal  teacher  evaluations 
carried  out  by  the  study  group  were  initiated  as  routine 
evaluations  according  to  school  board  policy.     Their  primary 
purposes  were:     to  improve  the  performance  of  the  teacher, 
to  award  a  permanent  contract,   to  provide  a  basis  for 
retention  in  position,  or  to  recommend  for  permanent 
certification.     As  a  result  of  the  evaluation,  most  teachers 
were  retained  in  their  present  position,   received  a 
permanent  contract  or  were  recommended  for  permanent 
certification.     Teacher  dismissal  was  cited  as  a  primary 
purpose  of  the  evaluation  by  4.1%  of  respondents  and  was 
reported  as  an  outcome  of  the  evaluation  by  4.8%  of 
respondents.     Formal  teacher  evaluation  was  rarely  used  as  a 
basis  for  transfer  or  promotion. 

8.  When  the  evaluation  was  initiated  as  a  routine 
evaluation  according  to  district  policy,   respondents  scored 
significantly  higher  on  the  completeness  measure  than  when 
they  did  not.     Other  initiating  factors  were  not  significant 
in  affecting  the  completeness  of  the  evaluation. 

When  the  primary  intended  purpose  of  the  last  formal 
evaluation  was  to  improve  the  performance  of  the  teacher,  or 
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to  record  the  performance  of  a  resigning  or  retiring 
teacher,   respondents  scored  significantly  higher  on  the 
completeness  measure  than  when  it  was  not.     When  the  purpose 
was  to  recommend   for  permanent  certification,  respondents 
scored  significantly  lower  on  the  completeness  measure. 
Other  intended  purposes  did  not  significantly  influence  the 
respondents'   score  on  the  completeness  measure. 

When,  as  a  result  of  the  evaluation,  an  inservice  plan 
was  developed  for  the  teacher,   respondents  scored  higher  on 
the  completeness  measure  than  when  no  plan  was  developed. 
If  the  teacher  was  recommended  for  permanent  certification 
as  a  result  of  the  evaluation,   the  respondents  scored 
significantly  lower  on  the  completeness  measure. 

9.     Experts  identified  17  of  the  37   items  on  the 
Modified  Acheson/Shinn  instrument  as  essential  to  assuring 
due  process  in  formal  teacher  evaluation.     In  essence,  the 
items  related  to  due  process  suggest  that:     (a)   standards  of 
performance  must  be  developed  for  the  teacher,    (b)  a  variety 
of  objective  data  must  be  collected,   (c)  a  post  conference 
must  be  held  with  the  teacher  to  discuss  the  data  and 
develop  recommendations  for  improvement,    (d)   a  plan  of 
assistance  must  be  developed  for  the  teacher,  and   (e)  the 
teacher  must  be  given  a  copy  of  the  evaluation  report  and 
the  opportunity  to  respond  in  writing. 

Experts  in  improvement  of  instruction  identified  25  of 
the  Modified  Acheson/Shinn  items  as  essential.     In  essence 
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the  improvement  of  instruction  criteria  would  require  that 
the  due  process  criteria  be  met,   and:      (a)   a  precon f erence 
must  be  held  to  discover  what  the  teacher  expected  the 
students  to  be  doing,  and  to  discern  what  problems  or 
concerns  the  teacher  had,    (b)  during  the  post  conference, 
greater  teacher  input  should  be  sought  and  appropriate 
praise  and  encouragement  given,   and   (c)  a  formal  appeal 
procedure  should  be  available  if  the  teacher  challenged  the 
report. 

10.  The  sixteen  items  identified  as  essential  by 
teachers  form  a  subset  of  the  improvement  of  instruction 
items.     The  most  notable  differences  fall  in  the  areas 
related  to  the  establishment  of  standards  and  the  collection 
of  objective  data.     Teachers  rate  these  items  lower  than 
either  of  the  groups  of  experts,  and  no  objective  data 
collection  strategies  were  rated  as  essential  by  teachers. 
In  addition,   teachers  rated  teacher  input  items  higher  than 
either  group  of  experts. 

11.  The  patterns  of  "yes"   responses  to  the  37   items  on 
the  Modified  Acheson/Shinn  instrument  were  somewhat  similar 
for  superintendents,  other  central  office  staff,  and 
principals;   however,   noteworthy  differences  occurred  on  ten 
i  tems : 

Less  than  half  the  superintendents,  but  more  than  three 
quarters  of  other  central  office  staff  and  principals  met  with 
the  teacher  prior  to  the  classroom  observation   (Item  3). 
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An  appeal  procedure   (Item  37)  was  more  likely  to  be 
made  available  when  the  evaluation  was  carried  out  by  a 
principal   (91.5%),   than  when  carried  out  by  other  central 
office  staff   (87.7%),  or  superintendents  (69%). 

Superintendents  and  other  central  office  staff  were 
more  likely  than  principals  to  record  student/teacher 
interactions   (Item  19),   record  impressions  about  the 
classroom  (Item  8),  examine  students'  work   (Item  24),  and 
examine  the  teacher's  planbook   (Item  21),     In  addition, 
superintendents  placed  a  greater  emphasis  upon  the 
examination  of  the  record  of  student  achievement   (Item  23). 

Principals,  more  than  other  central  office  staff,  and 
superintendents:     encouraged  teacher  input  in  the  post 
observation  conference   (Item  27),  gave  praise  and 
encouragement   (Item  30),  and  provided  the  opportunity  for 
the  teacher  to  respond  to  the  report  in  writing   (Item  36). 

Superintendents  developed  recommendations  to  help  the 
teacher  improve  instruction  (Item  32)  more  frequently  than 
other  central  office  staff  or  principals. 

12.     Few  of  the  respondents  met  the  minimum  criteria  to 
assure  due  process   (4.5%),  and  fewer  met  the  criteria  for 
the  improvement  of  instruction   (1.3%).     Essential  due 
process  items  not  used  in  the  last  evaluation  by  more  than 
50%  of  respondents  were: 

Item  34,   the  development  of  a  plan  of  assistance 
(superintendents,  other  central  office  staff,  and 
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pr inc  ipals  )  . 

Item  31,    the  recommendation  of  resources 
(superintendents  and  principals). 

Item  23,   the  examination  of  the  record  of  student 
achievement  (principals). 

Item  33,   the  development  of  recommendations  in 
cooperation  with  the  teacher  (superintendents). 

The  items  essential  for  the  improvement  of  instruction 
that  were  not  used  by  more  than  50%  of  respondents  included 
Items  34,   31,   and  33  as  noted  above  and: 

Item  29,   I  listened  more  than  I  talked 
(superintendents,  other  central  office  staff,  and 
principals ) . 

Item  3,   I  met  with  the  teacher  prior  to  the  observation 
(superintendents) . 

Twenty  nine   (8.3%)  of  the  respondents  met  the  minimum 
criteria  established  in  the  teacher  perception  subset.  The 
items  deemed  essential  by  teachers  that  were  not  used  by 
more  than  50%  of  respondents  were  Items  34,   31,   33,   and  3  as 
noted  above. 

Conclusions  and  Implications 

Demographic  Factors 

The  demographic  factors  that  relate  to  the  respondents 
and  that  are  correlated  with  a  significantly  more  complete 
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evaluation  are:     the  placing  of  a  high  priority  on  teacher 
evaluation,   and  high  levels  of  university  training.  In 
addition,  a  positive  correlation  exists  between  these  two 
factors . 

Experience   (except  in  the  case  of  superintendents),  the 
amount  of  inservice  training,   the  number  of  teachers 
supervised  and  the  adequacy  of  the  time  available  for 
teacher  evaluation  were  not  significantly  related  to 
completeness . 

This  implies  that,   in  the  selection  of  personnel  who 
will  have  a  major  responsibility  in  the  area  of  teacher 
evaluation,   the  primary  criteria  should  include  those 
related  to  a  commitment  to  teacher  evaluation  and  adequate 
related  university  training. 

Inservice  Training 

The  lack  of  a  significant  relationship  between 
inservice  training  and  the  completeness  of  the  evaluation  is 
almost  certainly  due  to  the  model  of  inservice  training 
commonly  in  use   in  Alberta,  namely  the  "single  shot,"  one  or 
two  day  lecture  or  workshop.     Studies  by  Joyce  and  Showers 
(1981)   indicate  that,   for  inservice  training  to  cause  a 
change  in  behavior,   five  training  elements  are  essential: 
theory,  demonstration,  practice,   feedback,  and  coaching. 
The  "single  shot"  strategy  can,   at  best,  encompass  the  first 
four  of  these  components.     This,   say  Joyce  and  Showers,  can 
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result  in  a  transfer  effect  of  up  to  10%  at  the  classroom 
level.     To  achieve  a  substantial  effect,  all   five  elements, 
used  in  sequence  are  required.     When  this  is  done,   a  90% 
transfer  effect  can  be  achieved. 

The  potential  for  the  successful   implementation  of  this 
model  in  Alberta   is  demonstrated  by  the  positive  outcomes 
reported  in  Townsend's   (1984)   study  of  the  implementation  of 
a  policy  on  teacher  evaluation  in  the  Lethbridge  School 
District  #  51. 

The  implications  of  the  present  lack  of  a  relationship 
between  inservice  training  and  completeness  of  evaluation, 
the  Joyce  and  Showers   (1981)   research,  and  the  Townsend 
(1984)   study  are  far  reaching.     They  suggest  that,   if  a 
change  in  the  evaluative  behavior  of  superintendents,  other 
central  office  staff,  and  principals  is  to  be  effected,  the 
whole  process  of  inservice  training  in  Alberta  should  be 
restructured  . 

Policy 

This  study  found  no  relationship  between  the  presence 
of  a  written  policy  on  formal  teacher  evaluation  and  the 
completeness  of  the  last  evaluation,   nor  was  the  level  of 
adherence  to  policy  significantly  related  to  completeness; 
however,   routine  evaluations,   initiated  in  accordance  with 
board  policy  tended  to  be  more  complete  than  those  not 
initiated  for  this  reason.     This  suggests  that  school  board 
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policies  may  serve  no  useful  purpose,  except  to  initiate 
teacher  evaluation. 

In  the  preliminary  stages  of  this  study  in  1983,  the 
researcher  collected  and  examined  the  teacher  evaluation 
policies  from  the  majority  of  Alberta  school  jurisdictions 
and  encountered  an  incredible  diversity  of  policies.  They 
ranged  from  complex  policies  that  incorporated  clearly 
delineated  strategies  and  instruments,   to  brief  statements 
of  intent;   from  policies  clearly  designed  to  assure  due 
process,   to  policies  that  obviously  contravened  recent  human 
rights  legislation.     This  informal  survey  confirmed  Reikie's 
(1977,  p.  v)   statement  that  "Overall,   there  were  few  school 
systems  with  clearly  structured  evaluation  policies  and 
practices.     Numerous  jurisdictions  appeared  to  have  no 
policies  regarding  many  facets  of  teacher  evaluations. 
Among  the  jurisdictions  there  was  a  wide  variety  of 
practices.     Few  practices  were  found  to  be  consistent  across 
Alberta ." 

The  absence  of  a  relationship  between  policy  and  the 
completeness  of  teacher  evaluation,   and  the  documented  lack 
of  consistency  of  teacher  evaluation  policies  across  the 
province  suggests  that  the  development  of  teacher  evaluation 
policies  at  the  local  board  level  may  be  an  ineffective 
mechanism  for  the  implementation  of  teacher  evaluation 
practices  across  the  province. 

The  State  of  Oregon,   U.S.A.,   has  adopted  a  statewide 
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model  which  prescribes  a  basic  form  of  teacher  evaluation, 

consistent  with  the  clinical  supervision  model,  yet  leaves 

local  boards  considerable  autonomy  within  its  basic 

guidelines.     That  model   is  delineated  in  Oregon  Revised 

Statute  342  .850: 

Teacher  evaluation;   form;  personnel  file  content.     ( 1 ) 
The  district  superintendent  of  every  school  district, 
including  superintendents  of  education  service 
districts,   shall  cause  to  have  made  at  least  annually 
but  with  multiple  observations  an  evaluation  of 
performance  for  each  probationary  teacher  employed  by 
the  district  and  at  least  biennially  for  every  other 
teacher.     The  purpose  of  the  evaluation  is  to  allow  the 
teacher  and  the  district  to  determine  the  teacher's 
development  and  growth  in  the  teaching  profession  and 
to  evaluate  the  performance  of  the  teaching 
responsibilities.     A  form  for  teacher  evaluation  shall 
be  prescribed  by  the  State  Board  of  Education  and 
completed  pursuant  to  rules  adopted  by  the  district 
school  board . 

(2)  (a)   The  district  school  board  shall  develop  an 
evaluation  process  in  consultation  with  school 
administrators  and  with  teachers.     If  a  school 
district's  teachers  are  represented  by  a  local 
bargaining  organization,   the  board  shall  consult  with 
the  teachers  belonging  to  and  appointed  by  the  local 
bargaining  organization  in  the  consultation  required  by 
this  paragraph. 

(b)  The  district  school  board  shall  implement  the 
evaluation  process  that  includes: 

(A)  The  establishment  of  job  descriptions  and 
performance  standards  which  include  but  are  not  limited 
to  items  included  in  the  job  description; 

(B)  A    preevaluat ion  interview  which  includes  but  is 
not  limited  to  the  establishment  of  performance  goals 
for  the  teacher,   based  on  the  job  description  and 
performance  standards; 

(C)  An  evaluation  based  on  written  criteria  which 
include  the  performance  goals;  and 

(D)  A  post-evaluation  interview  in  which   (i)  the 
results  of  the  evaluation  are  discussed  with  the 
teacher  and   (ii)  a  written  program  of  assistance  for 
improvement,   if  needed,   is  established. 

(c)  Nothing  in  this  subsection  is  intended  to  prevent  a 
district  from  consulting  with  any  other  individuals. 

(3)  Except  in  those  districts  having  an  average  daily 
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membership,   as  defined  by  ORS  327.006,  of  fewer  than 
200  students,   the  person  or  persons  making  the 
evaluations  must  hold  teaching  certificates.  The 
evaluation  shall  be  signed  by  the  school  official  who 
supervised  the  teacher.     A  copy  of  the  evaluation  shall 
be  delivered  to  the  teacher. 

(4)  The  evaluation  reports  shall  be  maintained   in  the 
personnel  files  of  the  district. 

(5)  The  evaluation  report  shall  be  placed  in  the 
teacher's  file  only  after  reasonable  notice  to  the 
teacher  . 

(6)  A  teacher  may  make  a  written  statement  relating  to 
any  evaluation,   reprimand,   charge,  action  or  any  matter 
placed   in  the  teacher's  personnel  file  and  such 
teacher's  statement  shall  be  placed  in  the  personnel 
file. 

(7)  The  personnel  file  shall  be  open  for  inspection  by 
the  teacher,   the  teacher's  designees  and  the  district 
school  board  and  its  designees.     District  school  boards 
shall  adopt  rules  governing  access  to  personnel  files, 
including  rules  specifying  whom  school  officials  may 
designate  to  inspect  personnel  files. 


Minimum  Criteria  for  Satisfactory  Evaluation 


As  the  findings  of  this  study  show,   few  respondents  met 
the  minimum  criteria  deemed  necessary  to  assure  due  process 
or  the  improvement  of  instruction  in  formal  teacher 
evaluation.     The  picture,  however,   is  not  as  bleak  as  it 
first  may  seem,   since  the  findings  also  reveal  that,  on 
average,  evaluators  carried  out  almost  three  quarters  of  the 
strategies  necessary  to  meet  those  requirements  and 
repeating  patterns  were  found  in  the  items  commonly  omitted. 
This  suggests  that  with  appropriate  training,  most  evaluator 
performance  could  be  readily  upgraded  to  meet  the  minimum 
criteria. 

The  strategies  deemed  essential  by  teachers  differed 
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from  those  of  experts  in   that  teachers  gave  lower  rankings 
to   items  related  to   the  establishment  of  standards  and  the 
collection  of  objective  data.     This  suggests  to  the 
researcher  that  teachers  are  anxious  about  evaluation  and 
particularly  those  processes  which  are  essential  in 
termination  cases,   namely  the  due  process   items.  The 
findings  of  this  study  suggest  that  the  great  majority  of 
formal  evaluations  carried  out  are  positive,   rather  than 
punitive  in  intent,   and  that  the  apparent  fears  exhibited  by 
teachers  are  generally  groundless.     However,   it  would  be 
folly  to  dismiss  those  fears,   since,   if  one  of  the  primary 
purposes  of  evaluation  is  to  improve   instruction,  a  negative 
perception  of  some  of  the  essential  processes  could 
seriously  inhibit  the  successful  implementation  of  any 
useful  program  of  evaluation. 

Recommendations 

As  a  result  of  the  findings  of  this  study,   and  the 
conclusions  drawn  therefrom,   it   is  recommended  that: 

1.  When  engaging  personnel  whose  major  responsibility 
will  be  the  evaluation  of  teachers,  school  boards  hire 
persons  who  place  a  high  priority  upon  teacher  evaluation 
and  who  have  related  university  training. 

2.  When  organizing   inservice  training  to  upgrade 
evaluators  in  their  jurisdictions,   school  boards  ensure  that 
the  model  of  inservice  training  used  is  one  that  results  in 
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a  change  in  behavior  at  the  classroom  level,   namely,  one 
that  includes  theory,  demonstration,   practice,   feedback,  and 
coaching . 

3.  That  school  boards  involve  classroom  teachers  in 
those  inservice  programs  so  that  they  become  more 
knowledgeable,  and  more  committed  to  the  processes  of  formal 
evaluation . 

4.  The  Government  of  Alberta  enact  legislation  to 
ensure  that  teacher  evaluation  is  carried  out  in  a  fair  and 
consistent  manner  throughout  the  province. 

Recommendations  for  Further  Study 

This  study  has  described  the  formal  teacher  evaluation 
practices  in  Alberta  during  the  period  September  1,   1983  to 
April  30,   1984,  as  reported  by  superintendents,  other 
central  office  staff,  and  principals.     In  addition,   it  has 
assessed  the  adequacy  of  the  current  evaluative  practices 
with  respect  to  minimum  criteria  in  the  areas  of  due  process 
and  the  improvement  of  instruction,  and  explored  the 
relationships  between  the  completeness  of  the  evaluation  and 
demographic  factors. 

One  limitation  of  this  study  is  that  it  has  described 
the  current  evaluative  practices  using  data  collected  from 
the  evaluators,   not  from  those  being  evaluated.  Other 
studies   (e.g.  Shinn,   1976)   suggest  that  the  perceptions  of 
teachers  and  administrators  may  be  different  and  so  future 
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studies  might  examine  the  teacher  perspective  in  more  depth. 

Data  collected  in  this  study  showed  that  teachers  did 
not  rate  the  establishment  of  standards  of  performance  and 
objective  data  collection  techniques  as  highly  as  did 
experts  in  due  process  and  improvement  of  instruction. 
Future  studies  might  focus  on  the  reasons  for  this 
difference  and  the  impact  of  inservice  training  upon  teacher 
perceptions. 

Joyce  and  Showers   (1981)  have  demonstrated  the  need  for 
a  model  of  inservice  training  that  includes  theory, 
demonstration,  practice,   feedback,  and  coaching.  They 
suggest  that  the  transfer  of  knowledge  into  actual  classroom 
practice  occurs  when  all  of  these  components  are  used  in 
sequence.     Future  studies  could  explore  the  impact  of  this 
inservice  model   in  improving  the  quality  of  teacher 
evaluation  in  Alberta. 

The  review  of  the  literature  revealed  that  early  in 
this  century,   researchers  discovered  that  the  evaluation  of 
teachers  had  a  dramatic  effect  on  student  outcomes.  No 
recent  studies  were  found  describing  the  relationship 
between  modern  teacher  evaluation  strategies  and  student 
performance.     Future  studies  might  examine  the  differences 
between  the  performance  of  students  in  classrooms  where  the 
teachers  were  evaluated,   as  opposed   to  those  where  no 
teacher  evaluation  took  place. 
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FORMAL  TEACIICR  EVALUATION  IN  ALBERTA 


This  questionnaire  surveys  current  teacher  evaluation  practice  in  Alberta.    Since  it  has  the  potential 
for  identifying  needs  and  proposing  solutions  for  an  issue  that  is  of  current  concern,  your  cooperation 
in  responding  to  it  is  appreciated.    The  results  of  the  study  (which  should  be  of  value  to  you  and  your 
colleagues)  will  be  disseminated  by  the  end  of  the  year. 

It  should  take  about  10  minutes  to  complete  the  questionnaire. 


For  the  purpose  of  this  research  project,  a  formal  teacher  evaluation  is  defined  as  one  that  includes  a 
uritten  report  containing  a  recormendation  or  ratine; . 


SECTION  I 

Please  respond  to  the  following: 

1.  Position  title  (check  one)      (1)  Superi ntendent Q  (2)  Other  Central  Office  Staff  Q  (3)  Principal  Q 

2.  How  many  years  experience  as  a  teacher  evaluator  do  you  have?   

3.  ilow  many  university  credit  courses  in  teacher  eval uation  have  you  completed?   

{U.S.  graduates  count  6  semester  hours  or  9  term  hours  as  1  course) 

4.  How  many  hours  of  in-service  training  in  teacher  evaluation  have  you  completed  in  the  last  5  years?   

5.  Respond  in  the  appropriate  position  category: 
Superintendent:    How  many  teachers  are  in  your  jurisdiction?   

(  ^/  ^^e  superintendent  of  more  than  one  jui^isdication,  please  itulicate  the  TOTAL  number  of  teachers 
you  are  responsible  for. ) 

Other  Central  Office:    How  many  teachers  are  in  your  area  of  responsibility?   

Principals:    How  many  teachers  are  in  your  school?   

6.  How  many  of  these  are  first  year  teachers?   

7.  How  many  are  experienced  teachers  new  to  your  school/system?   

8.  How  many  are  tenured?   

9.  Check  to  indicate  whether  your  jurisdiction  has  a  written  policy  on  fonnal  teacher  evaluation. 
(1)    Yes  O  (2)    Mo  O  (3)    Unsure  Q 

10.  In  relation  to  all  your  other  duties,  rate  (/>//  circlinij  the  appropriate  uivnber)  tlie  adequacy  of  the 
time  you  have  available  for  teacher  evaluation. 

Totally  adequate       12       3       4       5       Totally  inadequate 

11.  In  relation  to  all  of  your  duties,  rate  {b;i  circling  the  appropriate  number)  the  priority  of  formal 
teacher  evaluation. 

Very  High  Priority        12        3       4       5       Very  Low  Priority 


IF  YOU  HAVE  NOT  FORMALLY  EVALUATED  A  TEACHER  SINCE  SEPTEMBER  1,  1983,  PLEASE  STOP 
HERE  AND  RETURN  THIS  SURVEY  IN  THE  SELF-ADDRESSED  STAflPED  ENVELOPE.    THANK  YOU. 


Of  the  teachers  you  formally  evaluated  between  September  1  and  April  30  1984. 

12.  How  many  were  first  year  teachers?   

13.  How  many  were  experienced  teachers  new  to  your  school /system?   

14.  How  many  were  tenured? 
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SECTION  2 


RECALL  THE  LAST  FORMAL  TEACHER  EVALUATION  YOU  CARRIED  OUT  AND  RESPOND  TO  TllC 
REMAINING  QUESTIONS  WITH  RESPECT  TO  THAT  EVALUATION. 


1.  The  teacher  was:  (1)  A  first  year  teacher        (2)  Experienced  but  new  to  your  school /sys tern  Q) 
(3)  Tenured.  Q 

2.  Grade  level(s)  observed  

3.  Subject(s)  observed  [check  as  ai'\To\.viate)    (1)  MathQ  (2)  Language  Arts  Q 

(3)  Social  Studies  0(4)  Sciences  Q)        Other  Q  {Please  si^ecifj)   

4.  Number  of  separate  classroom  visitations   

5.  Minutes  spent  in  actual  classroom  observations   

6.  If  your  jurisdiction  has  a  written  evaluation  policy,  rate        civcHivj  the  ari  roi fi.itc  l-.") 
the  degree  to  which  you  adhered  to  policy. 

Strictly  to  the  Letter       1       2       3       4       5       Not  at  All 

7.  Rate,  (i-w  circling  the  aii^voiriate  number)  your  perception  of  the  teacher's  performance. 
Excellent       1      ^2       3       4       5  Unsatisfactory 

8.  Check  the  primary  intended  purpose  of  the  last  formal  evaluation. 
;.  (3  Recommending  permanent  certification 

■  -O  Awarding  a  permanent  contract 
c .  Q)  ^rQ^\6\r\q  a  basis  for  dismissal 

:'.  O  Recordi ng  the  performance  of  a  resigning  or  retiring  teacher 

(3  Providing  a  basis  for  transfer 

Q  Improving  the  performance  of  the  teacher 
'.Q  Providing  a  basis  for  promotion 
■•(3  Providing  the  basis  for  retention  in  position 

•  '.  (3  Determining  the  inservice  needs  of  the  teacher 

-'  '■O  Other  -  Please  specify  

9.  Check  the  primary  condition  that  was  involved  in  initiating  tlic  lost  formal  evaluation. 
j'-QParental  complaint 

".Q  Routine  evaluation  according  to  District  Policy 
•-^•QConcerns  arising  from  informal  observations 

Q  Teacher  request 
5- Q Need  for  documentation 

O Please  specify  

10.    As  a  result  of  the  evaluation: 

[check  all  that  ai'fly) 

■■  O^^^  teacher  was/will  be  recommended  for  pennanent  certification 

O^^^  teacher  was/will  be  awarded  a  pennanent  contract 
0.  O^he  teacher  was/will  be  dismissed 
V.  QThe  teacher  was/will  be  promoted 
^'  O^^^  teacher  was/will  continue  in  present  position 

O^"  in-service  plan  was/will  be  developed  for  the  teacher 

•  O^*^^  teacher  was/will  be  evaluated  by  another  evaluator 
■•  O^*^^  teacher  challenged  the  evaluation  report 

Qother  -  Please  specify   
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SECnO^  3 


Respond  to  the  fo11 


1. 

Yes  O 

No  O 

2  . 

Yes  Q 

No  O 

3. 

Yes  O 

No  O 

4. 

Yes  O 

No  O 

5. 

Yes  O 

no  O 

6. 

Yes  O 

No  O 

7. 

Yes  O 

No  O 

8 . 

Yes  O 

No  O 

9 . 

Yes  O 

No  O 

10. 

Yes  (J 

No  O 

11 . 

Yes  O 

No  O 

12. 

Yes  O 

No  O 

13. 

Yes  O 

No  O 

14. 

Yes  O 

No  O 

15. 

Yes  O 

No  O 

16. 

Yes  O 

No  O 

17. 

Yes  Q 

No  O 

18. 

Yes  O 

No  O 

19 . 

Yes  O 

No  O 

20. 

Yes  O 

No  O 

21. 

Yes  O 

No  O 

22  . 

Yes 

Mo 

23. 

YesQ 

Mo  O 

24. 

YesQ 

NoQ 

25. 

YesQ 

NoQ 

26 . 

Yes  O 

No  O 

2  7 . 

es  (J 

.10  Q 

28. 

Yes  Q 

No  O 

29. 

YesQ 

NoQ 

30. 

YesQ 

NoQ 

31. 

YesQ 

NoQ 

32. 

YesQ 

NoQ 

33. 

YesQ 

NoQ 

34. 

YesQ 

NoQ 

35. 

YesQ 

NoQ 

36. 

YesQ 

NoQ 

37. 

YesQ 

NoQ 

owing  statement  in  relation  to  the  last  formal  teacher  evaluation  you  conducted. 
he  teacher  had  a  written  job  description, 
tandards  of  performance  had  been  developed  for  the  teacher, 
met  with  the  t'-'cher  prior  to  the  observation. 

found  out  what  the  teacher  expected  the  students  to  be  doing  during  the  visit, 
found  out  what  concerns  or  problems  the  teacher  had. 

suggested  a  variety  of  observational  techniques  which  could  be  used  during  the  visit. 

he  teacher  had  input  in  determining  observational  strategies  and  the  type  of  data  to 
e  col lected. 

recorded  impressions  about  the  appearance  of  the  classroom. 

made  verbatim  notes  of  classroom  dialogue. 

recorded  data  on  the  teacher's  questions  for  later  analysis. 

recorded  data  on  students'  responses  to  teacher's  questions. 

recorded  data  on  student  on-task  behavior. 

recorded  data  on  the  patterns  of  students'  responses. 

made  audio  recordings  of  the  lesson. 

recorded  data  on  teacher  movement  patterns. 

recorded  data  on  student  movement  patterns. 

made  video  recordings  of  the  lessons. 

recorded  data  on  the  behavior  of  a  specific  student. 

recorded  subjective  data  on  the  quality  of  student/teacher  interaction, 
remained  in  the  classroom  for  at  least  30  minutes  when  1  visited, 
examined  the  teachers'  planbook. 

specifically  checked  that  the  program  was  in  accord  with  the  Alberta  Program  of  Studies, 
examined  the  Record  of  Student  Achievement, 
examined  student  work. 

met  with  the  teacher  after  the  observations, 
solicited  the  teaclier's  analysis  of  the  data. 

encouraged  the  teacher  to  make  inferences  and  tu  express  feelings  ond  opinions 
about  the  observational  data. 

encouraged  the  teacher  to  consider  alternative  teactiing  techniques  and  explanations 
of  classroom  events . 

listened  more  than  I  talked  during  tlie  post  observation  conference. 

gave  praise  and  encouragement  for  specific  growth  in  the  teacher's  teaching  skills. 

recommended  resources  such  as  books  and  training  programs  which  dealt  with  areas 
n  which  the  teacher  wished  to  improve. 

developed  recommendations  to  assist  the  teacher  improve  instruction. 

developed  recommendations  in  cooperation  with  the  teacher. 

developed  a  plan  of  assistance  for  the  teacher  as  an  outcome  of  the  evaluation. 

gave  the  teacher  a  copy  of  the  evaluation  report, 
he  teacher  was  given  the  opportunity  to  respond  in  writing  to  the  report, 
he  teacher  had  recourse  to  a  formal  appeal  procedure  if  he/she  challenged  the  report. 


f  you  have  any  comments  which  would  assist  the  researcher  in  describing  the  current  state  of  formal  evaluation 
n  Alberta,  please  use  the  next  page,  or  attach  additional  sheets  as  necessary. 

Thank  you  for  your  assistance  in  completing  this  questionnaire,  would  you  please  return  it  in  the  self-addressed 
stamped  enveloped  provided. 
Thank  you. 


FOR  OFFICE  USE  ONLY 

38  DPyQ  O 

39  :iyO  O 


A.N.  Duncan 

Lakeland  School  District  #5460 
Postal  Bag  Service  "A" 
BONNYVILLE,  Alberta     TOA  OLO 
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APPENDIX  B 
THE  MODIFIED  ACHESON/SH INN  INSTRUMENT 
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Listed  below  are  thirty  seven  statements  of  strategies  or  activities 
commonly  used  by  evaluators  in  the  formal  evaluation  of  teachers. 

(A  formal  evaluation  inoludea  a  writ  ten  report  containing  a  reaaimenda  tion 
or  rating. ) 


By  checking  the  appropriate  box,  please  rate  each  statement  with  respect 
to  it's  importance  in  a  fonnal  evaluation  designed  to  improve  instruction. 


he  teacher  had  a  written  job  description, 
tandards  of  performance  had  been  developed  for  the  teacher, 
met  with  the  teacher  prior  to  the  observation. 

found  out  what  the  teacher  expected  the  students  to  be  doing  during  the  visit, 
found  out  what  concerns  or  problems  the  teacher  had. 

suggested  a  variety  of  observational  techniques  which  could  be  used  during  the  visit. 

he  teacher  had  input  in  determining  observational  strategies  and  the  type  of  data  to 
be  collected. 

recorded  Impressions  about  the  appearance  of  the  classroom. 

made  verbatim  notes  of  classroom  dialogue. 

recorded  data  on  the  teacher's  questions  for  later  analysis. 

recorded  data  on  students'  responses  to  teacher's  questions. 

recorded  data  on  student  on-task  behavior. 

recorded  data  on  the  patterns  of  students'  responses. 

made. audio  recordings  of  the  lesson. 

recorded  data  on  teacher  movement  patterns. 

recorded  data  on  student  movement  patterns. 

made  video  recordings  of  the  lessons. 

recorded  data  on  the  behavior  of  a  specific  student. 

recorded  subjective  data  on  the  quality  of  student/teacher  interaction. 

remained  in  the  classroom  for  at  least  30  minutes  when  I  visited. 

examined  the  teachers'  planbook. 

specifically  checked  that  the  program  was  in  accord  with  the  Alberta  Program  of  Studies, 
examined  the  Record  of  Student  Achievement, 
examined  student  work. 

met  with  the  teacher  after  the  observations, 
solicited  the  teacher's  analysis  of  the  data. 

!  encouraged  the  teacher  to  make  inferences  and  to  express  feelings  and  opinions 
about  the  observational  data. 

encourarjed  the  teacher  to  consider  alternative  teaching  techniques  and  explanations 
f  classroom  events. 

listened  more  than  I  talked  during  the  post  observation  conference. 

gave  praise  and  encouragement  for  specific  growth  in  the  teacher's  teaching  skills. 


recommended  resources  such  as 
I  which  the  teacher  wished  to 


50oks  and  training  programs  which  dealt  with  areas 
nprove. 


developed  recommendations  to  assist  the  teacher  improve  instruction, 
developed  recommendations  in  cooperation  with  the  teacher. 

developed  a  plan  of  assistance  for  the  teacher  as  an  outcome  of  the  evaluation. 

gave  the  teacher  a  copy  of  the  evaluation  report, 
lie  teacher  was  given  the  opportunity  to  respond  in  writing  to  the  report, 
he  teacher  had  recourse  to  a  formal  appeal  procedure  if  he/she  challenged  the  report. 
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Listed  below  are  thirty  seven  statements  of  strategies  or  activities 

con¥nonly  used  by  evaluators  in  the  formal  evaluation  of  teachers. 

(A  formal  evaluation  inaludea  a  uritten  report  containing  a  recotmendation 
or  rating. ) 


By  checking  the  appropriate  box,  please  rate  each  statement  with  respect 
to  it's  importance  in  a  formal  evaluation  designed   to  meet  DUE  PROCESS 
requ  i  rements 


1.  0©®0 

2.  0000 

3.  0000 

4.  000© 

5.  000© 

6.  ©00© 

7.  ©00© 

8.  000© 

9.  ©0©© 

10.  ©00© 

11.  ©00© 

12.  ©00© 

13.  ©00© 

14.  ©00© 

15.  ©000 

16.  ©00© 

17.  ©00© 

18.  ©000 

19.  0000 

20.  000© 

21.  ©00© 

22.  ©00© 

23.  ©©0© 

24.  ©00© 

25.  0000 

26.  0000 

27.  0000 
2t3.  000© 

29-  0000 

30.  0000 

31.  000© 

3?-  0000 

33.  0O00 

34.  000© 

0000 

36-  ooo© 
OO00 


he  teacher  had  a  written  job  description, 
tandards  of  performance  had  been  developed  for  the  teacher, 
met  with  the  teacher  prior  to  the  observation. 

found  out  what  the  teacher  expected  the  students  to  be  doing  during  the  visit, 
found  out  what  concerns  or  problems  the  teacher  had. 

suggested  a  variety  of  observational  techniques  which  could  be  used  during  the  visit. 

he  teacher  had  Input  in  determining  observational  strategies  and  the  type  of  data  to 
e  collected. 

recorded  impressions  about  the  appearance  of  the  classroom. 

made  verbatim  notes  of  classroom  dialogue. 

recorded  data  on  the  teacher's  questions  for  later  analysis. 

recorded  data  on  students'  responses  to  teacher's  questions. 

recorded  data  on  student  on-task  behavior, 

recorded  data  on  the  patterns  of  students'  responses. 

made. audio  recordings  of  the  lesson. 

recorded  data  on  teacher  movement  patterns. 

recorded  data  on  student  movement  patterns. 

made  video  recordings  of  the  lessons. 

recorded  data  on  the  behavior  of  a  specific  student. 

recorded  subjective  data  on  the  quality  of  student/teacher  Interaction. 

remained  in  the  classroom  for  at  least  30  minutes  when  I  visited. 

examined  the  teachers'  planbook. 

specifically  checlced  that  the  program  was  in  accord  with  the  Alberta  Program  of  Studies, 
examined  the  Record  of  Student  Achievement, 
examined  student  work. 

met  with  the  teacher  after  the  observations, 
solicited  the  teacher's  analysis  of  the  data. 

encouraged  the  teacher  to  make  Inferences  and  to  express  feelings  and  opinions 
bout  the  observational  data. 

encouraged  the  teacher  to  consider  alternative  teaching  techniques  and  explanations 
f  classroom  events. 

listened  more  than  I  talked  during  the  post  observation  conference. 

gave  praise  and  encouragement  for  specific  growth  in  the  teacher's  teaching  skills. 

recommended  resources  such  as  books  and  training  programs  which  dealt  with  areas 
n  which  the  teacher  wished  to  improve. 

developed  recommendations  to  assist  the  teacher  improve  instruction. 

developed  reconmendations  in  cooperation  with  the  teacher. 

developed  a  plan  of  assistance  for  the  teacher  as  an  outcome  of  the  evaluation. 

gave  the  teacher  a  copy  of  the  evaluation  report, 
he  teacher  was  given  the  opportunity  to  respond  In  writing  to  the  report, 
he  teacher  had  recourse  to  a  formal  appeal  procedure  if  he/she  challenged  the  report. 
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30  April  1984 
Dear  Colleague, 

Teacher  evaluation  is  upon  us  1 

The  recent  announcement  by  the  Honourable  David  King  of  the  Provincial 
Evaluation  Policies  has  far  reaching  implications  for  school  jurisdictions 
and  educational  administrators. 

Given  the  very  natural  concerns  that  arise  following  the  publication  of 
such  a  policy,  it  is  clear  that  there  is  a  great  need  to  establish  "what  is" 
and  "what  should  be"  in  the  domain  of  teacher  evaluation.  To  this  end  we  are 
undertaking  a  study  to  document  the  teacher  evaluation  practices  currently 
used  by  administrators,  the  practices  preferred  by  teachers  and  the  practices 
necessary  to  meet  minimum  due  process  requirements.    It  is  anticipated  that 
the  study  will  generate  useful  information  for  the  formulation  of  sound, 
fair    teacher  evaluation  practices. 

As  part  of  this  study  we  are  soliciting  your  assistance  and  would  ask  that 
you  complete  and  return  the  attached  questionnaire  -  it  takes  about  10  minutes 
to  complete.    A  stamped,  addressed  envelope  is  provided. 

Individual  responses  will  remain  anonymous  and  confidential  and  a  summary  of 
the  research  will  be  disseminated  in  the  fall. 

Your  assistance  in  replying  as  soon  as  possible  is  appreciated. 

Sincerely, 

A.  N.  Duncan.  B.Ed. ,  M.Ed.  ,  A. CP. 

Teacher  Evaluation  Research  Project. 


140 


30  April  1984 


Dear  Colleague, 

RE:    Provincial  Evaluation  Policies  -  Teacher  Evaluation 


Before  rushing  headlong  into  hastily  conceived  schemes,  there  would 
seem  to  be  a  real  need  to  establish  "what  is"  and  "what  should  be" 
in  the  domain  of  teacher  evaluation.    To  this  end  the  Lakeland  School 
District  is  sponsoring  a  study  to  determine: 

(1)  Current  teacher  evaluation  practices  (including  the  influence 
of  training  and  experience  on  the  evaluation  strategies) 

(2)  evaluation  practices  preferred  by  teachers 

(3)  practices  necessary  to  meet  minimum  "due  process"  and 
"improvement  of  instruction"  requirements. 

We  are  soliciting  your  assistance  in  two  ways;  firstly  in  completing  and 
returning  the  enclosed  questionnaire  (it  takes  about  10  minutes)  and 
secondly  through  your  support  of  the  project  by  encouraging  your 
administrative  staff  to  respond.    -  The  questionnaire  was  sent  to  all 
Superintendents,  and  all  Assistants  listed  on  the  1983  C.A.S.S. 
membership  roster  and  a  sample  of  350  principals  from  around  the 
Province. 

Your  assistance  is  appreciated.  Thanks. 
Sincerely, 


A.N.  Duncan,  Coordinator 

Teacher  Evaluation  Research  Project 

AND/pr 


ENCL. 


141 


LAKELAND  PUBLIC  SCHOOL  DISTRICT  NO.  5460 


POSTAL  BAG  SERVICE  A 
BONNYVILLE,  ALBERTA  TOA  OLO 
PHONE;  (403)  826-3145 


DEPUTY  SUPERINTENDENT 


MR   W  HA^DUK 


14  May  1984 

Dr.  R.  Heyman,  Professor 

Dept.  of  Educational  Policy  &  Admin.  Studies 
The  University  of  Calgary 
2500  University  Drive  N.W. 
CALGARY,  Alberta 
JZr,  1N4 


Dear  Dr.  Heyman, 

We  are  conducting  a  study  into  the  practices  used  by  administrators 
in  the  formal  evaluation  of  teachers  and,  as  a  component  of  that 
research,  are  soliciting  the  assistance  of  acknowledged  experts 
in  delineating  essential  evaluation  strategies. 

You  were  identified  by  Dr.  Oliva,  Associate  Dean,  Faculty  of  Education, 
as  a  person  with  particular  expertise  in  teacher  evaluation  and  we 
would  be  most  grateful  if  you  assist  us  by  completing  and  returning 
the  attached  questionnaire  -  it  should  take  less  than  ten  minutes. 
A  self  addressed,  reply  paid  envelope  is  enclosed. 

Your  assistance  is  appreciated. 

Thank  you. 

Si  ncerely , 


A.N.  Duncan 

Teacher  Evaluation  Research  Project 

AND/pr 
Attachment 
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LAKELAND  PUBLIC  SCHOOL  DISTRICT  NO.  5460 


POSTAL  BAG  SERVICE  A 
BONNYVILLE.  ALBERTA  TQA  OLO 
PHONE:  (409)  3264145 


DEPUTY  SUPERINTENDENT 
Ml.  W.  HAYDUK 


23  May  1984  ■ 

Mr.  Richard  McNally 
A.S.T.A. 

12310  -  105  Avenue 
ECMONTOH,  Alberta 
T5N  0Y4 


Dear  Mr.  McNally, 

We  are  conducting  research  into  the  teacher  evaluation  practices 
currently  in  use  in  Alberta  and  are  soliciting  your  assistance 
because  of  your  acknowledged  expertise  in  the  domain  of  Due  Process 
in  teacher  evaluation. 

The  attached  questionnaire  lists  thirty  eight  evaluator  activities 
which  are  commonly  practiced.    You  are  asked  to  rate  each  of  these 
activities  with  respect  to  Due  Process  requirements.    This  will  enable 
us  to  identify  the  relative  importance  of  each  component  of  the 
evaluation  procedure  should  an  evaluation  ever  be  challenged  in  the 
courts  or  before  a  Board  of  Reference.    Your  assistance  in  completing 
and  returning  the  questionnaire  would  be  greatly  appreciated. 

Should  you  have  any  questions  regarding  this  research  please  do 
not  hesitate  to  call  me  (collect)  at  826-3145.    Thank  you. 

Sincerely, 


A.NTDuncan  B.Ed.,  M.Ed.,  A. CP. 
Teacher  Evaluation  Research  Project 


AND/pr 


Attachment 


May  20,  1984 


Dear  Colleague, 

WE  NEED  YOUR  HELP! 

With  the  pressures  that  administrators  face  at  this  time  of  year  and 

the  number  of  da  ed!  *@!  questionnaires  that  cross  one's  desk,  it 

is  probable  that  you  have  not  had  a  chance  to  get  to  the  survey  we 
sent  to  you  three  weeks  ago,  however,  we  need  YOUR  help. 

In  researching  "what  is"  and  "what  should  be"  in  the  domain  of  teachers 
evaluation,  we  are  attempting  to  provide  meaningful  data  for  the 
formulation  of  sound  policy  and  regulations,  and  to  provide  a  baseline 
for  assessing  the  impact  of  the  new  Provincial  Evaluation  Policies. 
We  cannot  do  this  without  your  help. 

Whether  or  not  you  have  conducted  any  teacher  evaluations,  won't  you 
please  take  a  few  minutes  now  to  complete  and  return  the  enclosed 
questionnaire.    A  reply  paid  envelope  is  provided. 

The  results  of  the  study  will  be  disseminated  in  the  fall. 

Thanks  for  you  assistance. 

Sincerely, 


A.N.  Duncan 

Teacher  Evaluation  Research  Project 
ph.  826-3145  (Bus.) 
826-6422  (Res.) 
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LAKELAND  SCHOOL  DISTRICT  #  5460 
Postal   Bag  Service  A 
Bonnyville,  Alberta. 
TOA  OLO 

18  June  1984 


Dear   Col  1 eague , 

In   any  research   project,    the  quality  or   value  of    the  study 
IS  dramatically  a-f-fected  by  the  response  rate.      It   is  our 
hope   that   we  can   maximize  the  value  o-f   our   study  on  Teacher 
Evaluation   by  achieving  as  high   a  response  rate  as  possible 
■from  the  teachers  surveyed. 

As  of   today  we  have  not  received  a  reply  from  you  on  the 
survey  we  mailed   last   month,    and   are  writing   to  ask   if  you 
would   take  a   few  minutes   to  complete  the  attached 
questionnaire  and  return   it    in   the  enclosed  prepaid 
envelope.      We  NEED  ^our   response  to  make  our  research 
meaningful      -     PLEASE  complete  and   return  the 
questionnaire. 

The  study  has  the  support  of   the  Planning  and  Research 
Branch   of   Alberta  Education.      The  findings  will  be 
disseminated    in   the  fall. 

PLEASE   RETURhJ   THE   QUESTIONNAIRE        -  THANKS 

5i ncerel y , 


Andv  Duncan 

Teacher    Evaluation   Research  Project 
ph.      826  3145  (bus) 
£i26   <b422  (res) 


145 


LAKELAND  SCHOOL  DISTRICT  #  5460 
Postal   Bag  Service  A 
Bonnyville,  Alberta. 
TOA  OLO 

18  June  1984 


Dear  Mr. 

In  any  research  project,   the  quality  or  value  o-f   the  study 
is  dramatically  affected  by  the  response  rate.      It   is  our 
hope  that  we  can  maximize  the  value  of   our  study  on  Teacher 
Evaluation  by  achieving  as  high  a  response  rate  as 
possi bl e. 


As  of   today  we  have  not  received  a  reply  from  you  on  the 
survey  we  mailed   last  month,   and  are  writing  to  ask   if  you 
would  take  a  few  minutes  to  complete  the  attached 
questionnaire  and  return   it   in  the  enclosed  prepaid 
envelope.     Even  if  you  have  not  evaluated  any  taacherm 
this  year,  or  if  you  are  the  principal  of  a  one  or  two  room 
school,   we  NEED  your  response     -     PLEASE  complete  and 
return   the  questionnaire. 


The  study  has  the  support  of   the  Planning  and  Research 
Branch  of   Alberta  Education  and  of   the  Conference  of 
Alberta  School    Superintendents.      The  findings  will  be 
disseminated   in   the  fall. 

PLEASE  RETURN  THE  QUESTIONNAIRE       -  THANKS 

Si ncerel y , 


Andy  Duncan 

Teacher   Evaluation  Research  Project 
ph.      826   3145  (bus) 
826   6422  (res) 
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SUM37 


Score   (Number  of   "yes"   responses)   on  Section  3  of 
the  ques tionna ire--The  Modified  Acheson/Shinn 
Instrument . 


PROCESS  -  Score  on  the  "due  process"  subset  of  the 
Acheson/Shinn  items. 

IMPRVT     -  Score  on  the   "Improvement  of   Instruction"  subset. 

TEACH       -  Score  on  the  "Teacher  Perception"  subset. 

Ql  -  Section  1  of  the  questionnaire. 

QI  -  Section  2  of  the  questionnaire. 

QUI         -  Section  3  of  the  questionnaire. 

Thus  Oil  8.1  refers  to  question  #  8,  part  1,  on  Section  2  of 
the  questionnaire. 


Value  labels: 

0  -     "No"  response 

1  -     "Yes"  response 

OR:     relate  to  the  item  bearing  that  number  on  the 
questionnaire.     (019  and  the  Multiple  Classification 
Analyses ) . 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY       01 18  I 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  7336  0000         24.3721  4,996  1     7488.3256  301 

1  1006  OOOO         21.8696  5.1923     1213.2174  46 

WITHIN  GROUPS  TOTAL  8342. OOOO         24.0403  5.0221     8701.5430  347 

ANALYSIS        OF         VARIANCE  * 

SUM  OF.  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  249.8922  1  249.8922  9.9078  .0018 

WITHIN  GROUPS  8701    5430  345  25.2219 

ETA    =        1671  ETA   SQUARED   =  0279 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY        01 18 . 2 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  6404. OOOO         23.8955  5.2763     743?. 0746  268 

1  1938.0000         24  5316  4.3760     1493.6709  79 

WITHIN  GROUPS   TOTAL  8342  OOOO  24.0403  5.0867     8926.7455  347 

•  ANALYSISOFVARIANCE  • 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG 

BETWEEN  GROUPS  24.6896  1  24.6896  .9542  .3293 

WITHIN  GROUPS  8926.7455  345  25.8746 

ETA   =      .0525         ET,^  SQUARED   =  .0028 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  Bv        0118  3 

----------------------  ANALYSIS        OF        VA  RIANCE 

VALUE      LABEL                                                   SUM  MEAN          STD  DEV     SUM  OF   SO  CASES 

0  7910.0OOO  23  9697           5.1132     8601,6970  330 

1  432.0000  25   4118            4.4449        316.1176  17 

WITHIN  GROUPS   TOTAL                                  8342. OOOO  24  0403            5.0842     8917.8146  347 

ANALYSIS       OF  VARIANCE 

SUM  OF  MEAN 

SOURCE                                            SQUARES                 D.F  SQUARE                   F  SIG. 

BETWEEN  GROUPS                             33.6205                    1  33.6205            1.3007  2549 

WITHIN  GROUPS                               8917   8146                 345  25  8487 
ETA    =        0613          ETA   SQUARED   =  0038 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY        01  IB  4 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STO  DEV     SUM  OF   SO  CASES 

0  8147  OCOO         23.9618  5.0905     8784.5029  340 

1  195. OOOO         27.8571  3.2367         62.857  1  7 

WITHIN  GROUPS   TOTAL  8342. OOOO  24.0403  5.0640     8847.3601  347 

ANALYSIS       OF        VARIANCE  • 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  104.0751  1  104   0751  4.0584  .0447 

WITHIN  GROUPS  8847   3601  345  25.6445 

ETA    =      ,1078  ETA   SQUARED    =  0116 
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CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY        01 18  5 

----------------------        ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  8078. OOOO         23.9703  5    1024     8747.7033  337 

1  264. OOOO         26.4000  4.0332        146.40OO  10 

WITHIN  GROUPS   TOTAL  8342. OOOO  24.0403  5  0774      8894.1033  347 

AN  ALVSIS       OF        VARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  O.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  57   3319  1  57   3319  2.2239  ,1368 

WITHIN   GROUPS  8894    1033  345  25 . 7800 

ETA    =        0800  ETA   SQUARED   =  0064 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY        01 18 . 6 

---------  --------        ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  5339. OOOO         23.3144  5.1612     6073.3624  229 

1  30O3.00O0         25  4492  4,6439     2523.1949  118 

WITHIN  GROUPS   TOTAL  8342,0000  24,0403  4,9917      8596,5574  347 

ANALYSIS        OF        VARIANCE  * 

SUM  OF  MEAN 

SOURCE  SQUARES  D,F,  SQUARE  F  SIG 

BETWEEN  GROUPS  354   8778  1  354.8778  14,2421  , 0002 

WITHIN  GROUPS  8596,5574  345  24  9176 

ETA    =      ,1991  ETA   SQUARED   =  ,0396 
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CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01 18  7 

ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

n  8193  OOOO         24  0264  5.0657     8724  7625  34  1 

1  149. OOOO         24  8333  6.6758       222.8333  6 

WITHIN   GROUPS    TOTAL  8342   OOOO  24.0403  5.0926      8947.5958  347 

ANALYSIS       OF  VARIANCE 

SUM  OF  MEAN 

SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  3.8394  1  3.8394  .1480         . 7007 

WITHIN  GROUPS  8947  5958  345  25.9351 

ETA    =        0207  ETA   SQUARED   *      . 0004 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01  IS . 8 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF   SO  CASES 

0  6819,0000         24.0106  5.2523     7806.9683  284 

1  1523. OOOO         24.1746  4  2938      1143.0794  63 

WITHIN  GROUPS  TOTAL  8342. OOOO         24.0403  5.0933     8950.0477  347 

ANALYSIS       OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F  SQUARE  F  SIG. 

BETWEEN  GROUPS  1    3875  1  1.3875  .0535  8172 

WITHIN  GROUPS  8950  0477  345  25  9422 

ETA    -        0124  ETA   SQUARED    =  0002 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY  0II8.9 

ANALYSIS        OF  VARIANCE 
VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   50  CASES 

0  7623  OOOO         23.9717  5.0262     8014.7453  318 

1  719. OOOO         24  7931  5.7282       918.7586  29 

WITHIN  GROUPS  TOTAL  8342. OOOO         24.0403  5.0886     8933.5039  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  O.F  SQUARE  F  SIG 

BETWEEN  GROUPS  17.9313  t  17  9313  6925  .4059 

WITHIN  GROUPS  8933  5039  345  25.8942 

ETA    -        0448  ETA   SQUARED   =      . 0020 


CRITERION  VARIABLE  SUM37 

BROKEN  DOWN  BY       01 18. 10 

ANALYSIS       OF  VARIANCE 
VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF  SO  CASES 

0  7123. OOOO         24.2279  5.1  M6     7655.7313  294 

1  1219. OOOO         23.0000  4.8596      1228. OOOO  53 

WITHIN  GROUPS  TOTAL  8342. OOOO         24.0403  5.0744     8883.7313  347 

ANALYSIS       OF        VARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  67.7039  1  67  7039  2.6293  .1058 

WITHIN  GROUPS  8883.7313  345  25.7499 

ETA    =      .0870  ETA   SQUARED    =  0076 
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CRITERION   VARIABLE  5UM37 

BROKEN   DOWN   BV  01 19  1 

----------------------  ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD   DEV  SUM  OF    SO  CASES 

0  7844. OOOO  23.9878  5.0616  8351.9511  327 

1  498  OOOO  24.90OO  5.543  1  583.8OO0  20 

WITHIN  GROUPS   TOTAL  8342  OOOO  24  0403  5.0893  8935.7511  347 


SOURCE 

BETWEEN  GROUPS 
WITHIN  GROUPS 


SUM  OF 
SQUARES 

15 . 684  1 

8935    75  11 

ETA    =  ,'0419 


MEAN 

OF  SQUARE 

1  15.5841 
345  25.9007 
ETA   SQUARED   =  .0018 


F  SIG. 
.6055  .4370 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY  0II9.2 


VALUE  LABEL 

0 

WITHIN  GROUPS  TOTAL 


ANALYSIS  OF  VARIANCE 
SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 


2315. OOOO 
6027 .OOOO 


23 . 15O0 
24 . 4008 


5.8558  3394.7500 
4.7048  5445.3198 


100 
247 


5.0620  8840.0698 


ANALYSIS       OF  VARIANCE 


SOURCE 

BETWEEN  GROUPS 
WITHIN  GROUPS 


SUM  OF  MEAN 

SQUARES                  OF.  SQUARE 

^1'   3653                    1  111.3653 

8840.0698                345  25.6234 

ETA    =        1115  ETA    SQUARED  -  .0124 


F  SIG. 
4.3462  .0378 
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CPITERION  VARIABLE  5UM37 
BROKEN  DOWN  By       01 19  3 

ANALYSIS        OF  VARIANCE 
VALUE      LABEL  SUM  MEAN  STD   DEV      SUM  OF    SO  CASES 

0  7700.0000         23  9  130  5  0860     8303.5652  322 

1  642.0O00         25  6800  4.8966       575.440O  25 

WITHIN  GROUPS    TOTAL  8342  OOOO  24  0403  5  0731      8879. 0052  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  0  F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  72  4299  1  72  4299  2.8143  .0943 

WITHIN  GROUPS  8879.0052  345  25  7362 

ETA    ^        09OO  ETA   SQUARED   ^        008  1 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01 19  4 

----------------------  ANALYSIS       OF  VARIANCE 

VALUE     LABEL                                                SUM  MEAN         STD  DEV     SUM  OF  SO  CASES 

0  7637. OOOO  24.0157           5.0314     8024.9214  318 

1  705. OOOO  24  3103           5.7452       924.2069  29 

WITHIN  GROUPS  TOTAL                                8342. OOOO  24.0403           5.0931     8949.1283  347 

*  ANALYSISOFVARIANCE 

SUM  OF  MEAN 

SOURCE                                               SQUARES                  D.F  SQUARE                     F  SIG. 

BETWEEN  GROUPS                                2   3069                    1  2.3069              .0889  .7657 

WITHIN  GROUPS                            8949    1283               345  25  9395 

ETA   =      .0161  ETA   SQUARED   =  0003 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY        01 19  5 

------------  ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN         STD  DEV     SUM  OK   SO  CASES 

0  7178. OOOO  24.2500  5.1917     7951. 5000  296 

1  1164. OOOO         22.8235  4  2G95       911.4118  51 

WITHIN  GROUPS   TOTAL  8342. OOOO  24.0403  5.0685     8862.9118  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  88   5234  1  88.5234  3.4459  .0643 

WITHIN  GROUPS  8862.9118  345  25.6896 

ETA    =      .0994  ETA   SQUARED   =  .0099 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  8V  0II9.6 

ANALYSIS       OF  VARIANCE 
VALUE     LABEL  suM  MEAN         STD  DEV     SUM  OF   SO  CASES 

0  7949. OOOO         24.0879  5.1099     8590.45  15  330 

1  393. OOOO  23.1176  4.6487        345  7647  17 

WITHIN  GROUPS  TOTAL  8342. OOOO         24.0403         ' 5  0894 "sgSS "2 1 62  347 

ANALYSIS       OF        VARIANCE  ♦ 

SUM  OF  MEAN 
SQUARES  OF  SQUARE  F  SIG 

BETWEEN  GROUPS  ,5  2189  1  ,5.2189  .5876  .4439 

WITHIN  GROUPS  8936   2,62  345  25  9021 

ETA    -        0412  ETA   SQUARED    =        OO 1 7 
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CRITERION   VARIABLE  SUM37 

BROKEN  DOWN   BY  011101 

---  ------------------         ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  7  203  0000         24  4169  4  9938     7331.7153  295 

1  1139. OOOO         2  1   9038  5.1269      1340.5192  52 

WITKIN  GROUPS    TOTAL  8342. OOOO  24.0403  5.0137      8672.2345  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D   F  SQUARE  F  SIG. 

BETWEEN  GROUPS  279   2007  1  279   2007  11.1072  OOIO 

WITHIN  GROUPS  8672,2345  345  25.1369 

ETA    =      .1766  ETA   SQUARED   =  .0312 


CRITERION  VARIABLE  SUM37 

BROKEN  DOWN  BY       01 110. 2 

--- ANALYSIS       OF  VARIANCE 

VALUE      LABEL                                                   SUM  MEAN          STD  DEV     SUM  OF   SO  CASES 

0  6733. OOOO  23.9609           5.1722     7490.5694             28  1 

1  1609.0000  24.3788           4.7256      1451.5303  66 

WITHIN  GROUPS   TOTAL                                  8342.0000  24  0403            5.0911      8942.0997  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 

SOURCE                                            SQUARES                 D.F  SQUARE                    F  SIG. 

BETWEEN  GROUPS                                9.3355                    1  9.3355              .3602  .5488 

WITHIN  GROUPS                             8942.0997                345  25.9191 

ETA    =        0323  ETA   SQUARED    =  0010 
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CRITERION   VARIABLE  SUM37 

BROKEN  DOWN  BY        01  I  10  3 

-----   ANALYSIS        OF  VARIANCE 

VALUE      LABEL                                                   SUM  MEAN          STD  DEV     SUM  OF    50  CASES 

0  7818. OOOO  23  9816           5.1231     8529.8896  326 

1  524  OOOO  24  9524           4.4886       402.9524               2  1 

WITHIN  GROUPS   TOTAL                                 8342  OOOO  24  0403            5.0884      8932.8420  347 

ANALYSIS        OF        VARIANCE  • 

SUM  OF  MEAN 

SOURCE                                               SQUARES                  OF  SQUARE                     F  SIG 

BETWEEN  GROUPS                                18   5932                     1  '            18   5932                7181  3974 

WITHIN  GROUPS                             8932   8420                345  25.8923 

ETA    -       045S  ETA   SQUARED   =        002  1 


CRITERION  VARIABLE  SUM37 

BROKEN  DOWN  BY        Oil  10.4 

----------------  -        ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF   SO  CASES 

0  8158. OOOO         23  994  1  5.0758     8733.9882  340 

1  184. OOOO         26.2857  5.4989       181.4286  7 

WITHIN  GROUPS  TOTAL  8342. OOOO  24.0403  5.0835     8915.4168  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  36  0184  1  36.0184  1.3938  .2386 

WITHIN  GROUPS  8915.4168  345  25.8418 

ETA    =        0634  ETA   SQUARED    -  OO40 
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CRITERION    VARIABLE  SUM37 

BROKEN   DOWN   BY        Oil  10  5 

--    --     --     -  -     -     ------------         ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM                MEAN          STD  DEV     SUM  OF    SO  CASES 

0  3969  OOOO         24  2012           5.5740     5064.3598  164 

1  4  37  3  OOOO         23.8962           4.6166     3879.0273  183 

WITHIN   GROUPS    TOTAL                                    8342  OOOO          24.0403             5   09 1 4      8943.3871  347 

ANALYSIS        OF        VARIANCE  ♦ 

SUM  OF  MEAN 

SOURCE                                               SQUARES  D.F                SQUARE                     F  SIG. 

BETWEEN  GROUPS                                  8  048 1  1                   8.0481               .3105  .5778 

WITHIN  GROUPS                            8943.3871  345               25  9229 

ETA    -       0300  ETA   SQUARED   =■  0009 


CRITERION  VARIABLE  SUM37 

BROKEN  DOWN  BY       Oil  10. S 

ANALYSIS       OF  VARIANCE 
VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  7142. OOOO         23.5710  5.0652     7748.2244  303 

1  1200. OOOO         27.2727  3.9671       676.7273  44 

WITHIN  GROUPS  TOTAL  8342.0000         24.0403  4.9417     8424.9517  347 

•  ANALVSISOFVARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F  SQUARE  F  SIG. 

BETWEEN  GROUPS  526.4835  1  526.4835         2  1.5594  OOOO 

WITHIN  GROUPS  8424.9517  345  24.4201 

ETA    =      .2425         ETA  SQUARED   =  0588 
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CRITERION  VARIABLE  5UM37 

BROKEN  DOWN  BY        OHIO  7 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  7846. OOOO         23.9939  5.1344     8593.9878  327 

1  496. OOOO         24  80OO  4  2624       345.2000  20 

WITHIN  GROUPS  TOTAL  8342  OOOO         24.0403  5.0903     8939.1878  347 

ANALYSIS        OF        VARIANCE  • 

SUM  OF  MEAN 

SOURCE  SQUARES  D   F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  12   2474  1  12.2474  .4727  .4922 

WITHIN  GROUPS  8939    1878  345  25  9107 

ETA    =      .0370  ETA   SQUARED   =      .  O0 1 4 


CRITERION  VARIABLE  SUM37 

BROKEN  DOWN  BY        OHIO  8 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF   SO  CASES 

0  8257. OOOO         24.0029  5.0890     8882.997  1  344 

1  85. OOOO         28.3333  2.5166         12.6667  3 

WITHIN  GROUPS  TOTAL  8342. OOOO         24.0403  5.0778     8895.6638  347 

ANALYSIS        OF        VARIANCE  • 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  55   7714  1  55   7714  2.1630  .1423 

WITHIN  GROUPS  8895.6638  345  25.7845 

ETA   =      .0789         ETA  SQUARED   -  0O62 
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CRITERION   VARIABLE  SUM37 

BROKEN   DOWN   BY  01110,9 

--         -         --  -------------        ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD   DEV      SUM  OF    SO  CASES 

0  7363. OOOO         23  9837  5.0846     7910. 9186  307 

1  979.0000  24   4750  5    1440      1031.9750  40 

WITHIN  GROUPS   TOTAL  8342. OOOO  24   0403  5  09 1 3     8942.8936  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  OF  SQUARE  F  SIG. 

BETWEEN  GROUPS  8.5416  1  8.5416  .3295  5663 

WITHIN   GROUPS  8942    8936  345  25.9214 

ETA    ^        0309  ETA   SQUARED   =  .0010 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY        01  I  I  1 

ANALYSIS       OF  VARIANCE 
VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF    SO  CASES 

0  4553. OOOO         22.8794  5.2287     5413.1055  199 

1  3789. OOOO  25.6014  4.4489     2909.4797  148 

WITHIN  GROUPS   TOTAL  8342. OOOO         24.0403  4.9116     8322.5853  347 

ANALYSIS       OF        VARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  628.8499  1  628.8499         26.0680  OOOO 

WITHIN  GROUPS  8322.5853  345  24.1234 

ETA    =        2650  ETA   SQUARED    =  0703 
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CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01  I  12 

-----------   ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STO  DEV     SUM  OF   SO  CiiSES 

0  3144. OOOO         22    1408  5.4922     4253.1031  142 

1  5198. OOOO         25  356  1  4  3335     3831.0049  205 

WITHIN  GROUPS   TOTAL  8342  OOOO  24.0403  4.8407     8084.1880  347 

ANALYSIS        OF        VARIANCE  • 

SUM  OF  MEAN 

SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  867.2472  1  867.2472  37.0106  OOOO 

WITHIN  GROUPS  8084    1880  345  23  4324 

ETA    =        3  113  ETA   SQUARED   =  .0969 


CRITERION  VARIABLE  5UM37 
BROKEN  DOWN  BY       Oil  13 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STO  DEV     SUM  OF   SO  CASES 

0  1655  OOOO         20  9494  5.9397     2751,7975  79 

1  ,  6687. OOOO         24.9515  4.4226     5222.3694  268 

WITHIN  GROUPS   TOTAL  8342,0000  24.0403  4.8077      7974    1669  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F  SQUARE  F  SIG. 

BETWEEN  GROUPS  977.2683  1  977   2683  42.2812  OOOO 

WITHIN  GROUPS  7974.1669  345  23.1135 

ETA    =      ,3304  ETA   SQUARED    =  .1092 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY        Oil  14 


VALUE  LABEL 

0 


N    A    L  V 
MEAN 


132  1  OOOO  2  1  3065 
702  1.0000         24  6351 


SIS       OF  VARIANCE 
STD  DEV      SUM  OF    SO  CASES 


6 . 6937  2733  1774 
4.4619  5654.0491 


62 
285 


THIN   GROUPS  TOTAL 


8342  OOOO 


4.9306  8387.2265 


SOURCE 

BETWEEN  GROUPS 
WITHIN  GROUPS 


SUM  OF 
SQUARES 

564  2086 

8387  2265 

ETA    =        25  1  1 


MEAN 

D.F  SQUARE 

1  564.2086 
345  24  3108 

ETA   SQUARED   ^  0630 


F 

23.208  1 


SIG. 
OOOO 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01  I  15 


VALUE  LABEL 

O 


WITHIN  GROUPS  TOTAL 


1039. OOOO 
7303  OOOO 


ANALYSIS       OF  VARIANCE 
MEAN  STD  DEV     SUM  OF    SO  CASES 


20. 3725 
24 . 6723 


6 . 4  186 
4 . 5425 


2059 . 92 16 
6087  2  128 


5  1 
296 


4 .8595     8147 .  1344 


A   N   A   L    Y  S 


VARIANCE 


SOURCE 

BETWEEN  GROUPS 
WITHIN  GROUPS 


SUM  OF 
SQUARES 

804  3008 

8  147  1344 

ETA   =  2998 


MEAN 

D.F  SQUARE 

1  804 . 3008 

345  23  6149 

ETA   SQUARED   =  .0899 


F 

34 .059' 


SIG, 
OOOO 
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CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY        01  I  16 

--  -  ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF    SO  CASES 

0  5168.0000         22  4696  4  9147     5531.2870  230 

1  3174. OOOO  27    1282  3  8697      1737.0769  117 

WITHIN  GROUPS   TOTAL  8342  OOOO  24  0403  4 . 5900     7268.3639  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  5IG. 

BETWEEN  GROUPS  1683  0713  1  1683. 0713  79  8886  OOOO 

WITHIN  GROUPS  7268.3639  345  2 1  0677 

ETA    =        4336  ETA   SQUARED   =  1880 


CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY  01117 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF   SO  CASES 

0  5082. OOOO         22.5867  5.0465     5704 . 5600  225 

1  3260. OOOO         26.7213  3.9569      1894.5246  122 

WITHIN  GROUPS  TOTAL  8342.0000         24.0403  4.6932     7599.0846  347 

*  ANALYSISOFVARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  OF  SQUARE  F  SIG. 

BETWEEN  GROUPS  1352.3506  1  1352. 3506         6  1.3970  .OOOO 

WITHIN  GROUPS  7599.0846  345  22.0263 

ETA    =      .3887  ETA   SQUARED    =  1511 
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CRITERION    VARIABLE  5UM37 
BROKEN   DOWN   By        QI  I  I  8 

-    -    -----    -------------         ANALYSIS        OP  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  8  12  OOOO         20.8205  7.16G7      1951.7436  39 

1  7  530  OOOO  24.4481  4.6170     6544.1638  308 

WITHIN  GROUPS   TOTAL  8342. OOOO  24.0403  4.9624     8495.9124  347 

ANALYSIS        OF  VARIANCE 

SUM  OF'  MEAN 
SOURCE  SQUARES  O.F  SQUARE  F  SIG. 

BETWEEN  GROUPS  455.5227  I  455  5227  18  4978  OOOO 

W'THIN  GROUPS  8495.9124  345  24  6258 

ETA    =        2256  ETA   SQUARED    =  0509 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY        01  I  19 

----------------------  ANALYSIS       OF  VARIANCE 

VALUE      LABEL                                                   SUM  MEAN          STD  DEV     SUM  OF    SO  CASES 

0  3309. OOOO  22.0600           5.3360     4242.4600  150 

1  5033. OOOO  25  5482           4.3288     3672.7919  197 

WITHIN  GROUPS   TOTAL                                  8342. OOOO  24.0403            4.7899     7915.2519  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 

SOURCE                                               SQUARES                  D.F  SQUARE                     F  SIG. 

BETWEEN  GROUPS                          1036.1833                    1  1036    1833         45.1638  OOOO 

WITHIN  GROUPS                             7915.2519                345  22  9428 

ETA   =       3402         ETA   SQUARED   =  .1158 
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CRITERION   VARIABLE  5UM37 
BROKEN  DOWN  8V        QUI  10 

------    ----------------        ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD   DEV      SUM  OF    SO  CASES 

0  2467  OOOO         20.7311  4.3933     2825.3950  119 

1  5875. OOOO         25.7675  4,2720     4142.6798  228 

WITHIN  GROUPS    TOTAL  8342. OOOO  24   0403  4   494 1      6968.0748  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  1983.3604  1  1933.3604         98.1992  OOOO 

WITHIN  GROUPS  6968.0748  345  20.1973 

ETA    =        4707  ETA   SQUARED   =      .22  16 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01  I  I  1 1 

------  ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STO  DEV     SUM  OF   SO  CASES 

0  2369. OOOO         20.9646  4.886  1     2673.8534  113 

1  5973. OOOO         25.5256  4.4876     4692.3462  234 

WITHIN  GROUPS   TOTAL  8342. OOOO  24.0403  4.6207      7366.2046  347 

•  ANALYSISOFVARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  1585.2306  1  1585.2306         74.2451  .OOOO 

WITHIN  GROUPS  7366   2046  345  21  3513 

ETA    =        4208  ETA   SQUARED   -        177  1 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  Bv       01 II 12 

---    --    -     ------------        ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  OEV     SUM  OF   50  CASES 

0  888.00O0  19  7333  6.2719      1730. 8000  45 

1  7454  OOOO         24  6821  4.5610     6261.4834  302 

WITHIN  GROUPS    TOTAL  8342. OOOO  24.0403  4.8131      7992.2834  347 

ANALYSIS        OF        VARIANCE  • 

SUM  OF  MEAN 
SOURCE  SQUARES  OF  SQUARE  F  SIG. 

BETWEEN  GROUPS'  959    1517  1  959.1517  4  1.4034  OOOO 

WITHIN  GROUPS  7992   2834  345  23.1660 

ETA    =        3273  ETA   SQUARED    =      . 1072 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BV       01 1 113 

------------   ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         5T0  OEV     SUM  OF  SO  CASES 

0  3645. OOOO         21.6964  4.9011     4011.5179  168 

1  4697. OOOO         26.2402  4.2072     3150.6704  179 

WITHIN  GROUPS  TOTAL  8342.0000         24.0403  4.5563     7162.1882  347 

*  ANALYSISOFVARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  1789.2469  1  1789.2469         86.1874  OOOO 

WITHIN  GROUPS  7162.1882  345  20 . 7600 

ETA   =      .4471  ETA  SQUARED   =  1999 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY  011114 

-----------    -----  -  ANALYSIS        OF  VARIANCE 

VALUE      LABEL                                                      SUM  MEAN          5TD   DEV      SUM  OF    SO  CASES 

0  8182. OOOO  23  9941            5.0927     8817.9883             34  1 

1  160,0000  2S  6667           4.2740         91.3333  6 

WITHIN  GROUPS  TOTAL                                8342  OOOO  24.0403           5.0817     8909.3216  347 

ANALYSIS        OF         VARIANCE  • 

SUM  OF  MEAN 

SOURCE                                               SQUARES                  OF  SQUARE                     F  SIG. 

BETWEEN  GROUPS                             42    1136                    1  42.1135            1.6308  .2025 

WITHIN  GROUPS                             8909   3216                345  25  824  1 

ETA    =        0686  ETA   SQUARED    =  0047 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY        01  I  I  15 

ANALYSIS       OF  VARIANCE 
VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  4041. OOOO         21.7258  4.7812     4229.0161  186 

1  4301. OOOO         26.7143  4.0116     2574.8571  161 

WITHIN  GROUPS   TOTAL  8342. OOOO  24.0403  4.4409     6803.8733  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  2147.5619  1  2147.5619        108.8952  OOOO 

WITHIN  GROUPS  6803.8733  345  19  7214 

ETA   =      .4898         ETA  SQUARED   =  2399 
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CRITERION   VARIABLE  SUM37 
BROKEN   DOWN   BY        01  I  I  16 

-    -         ------------------        ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  4190. OOOO         22.1693  4.9274     4564.5820  189 

1  4152  OOOO         26  2785  4   3228     2933.7468  158 

WITHIN  GROUPS   TOTAL  8342. OOOO         24.0403  4.6620     7498.3288  347 

ANALYSIS        OF        VARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  OF  SQUARE  F  SIG. 

BETWEEN  GROUPS  1453.1063  1  1453    1063         66.8578  OOOO 

WITHIN  GROUPS  7498.3288  345  21  7343 

ETA    =      . 4029  ETA   SQUARED   ^  1623 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY        01  I  I  17 

----------------------        ANALYSIS        OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF  SO  CASES 

0  8310. OOOO         24  0173  5.0756     8887.8960  346 

1  32.0000         32. OOOO  .OOOO  .0000  1 

WITHIN  GROUPS   TOTAL  8342. OOOO         24.0403  5.0756     8887.8960  347 

•  ANALYSISOFVARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG 

BETWEEN  GROUPS  63.5392  1  53  5392  2.4664  .1172 

WITHIN  GROUPS  8887.8960  345  25  7620 

ETA    «      .0843  ETA   SQUARED    =        007  1 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY       01  I  I  18 

----------------------  ANALYSIS        OF  VARIANCE 

VALUE      LABEL                                                   SUM  MEAN          STD  DEV     SUM  OF   SO  CASES 

0  3356  0000  22.5497           5.3028     4780.3275  171 

1  4486,0000  25.4886            4.4220     3421.9773  176 

WITHIN  GROUPS   TOTAL                                 8342  OOOO  24  0403            4   8759     8202.3048  347 

ANALYSIS        OF        VARIANCE  • 

SUM  OF  MEAN 

SOURC;                                               SQUARES                  OF  SQUARE                     F  SIG. 

BETWEEN  GROUPS                           749    1304                    t  749    1304         31   5094  .OOOO 

WITHIN  GROUPS                             8202   3048                345  23.7748 

ETA    -        2893  ETA   SQUARED    =  0837 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01  I  I  19 

 ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF  SO  CASES 

0  733. OOOO         19.8108  7.1680     1849.6757  37 

1  7609  OOOO         24.5452  4.537  1     6360.8677  310 

WITHIN  GROUPS  TOTAL  8342.0000         24.0403  4.8784     8210.5434  347 

ANALYSIS        OF        VARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  O.F  SQUARE  F  SIG. 

BETWEEN  GROUPS  740.8917  1  740.8917  31.1316  OOOO 

WITHIN  GROUPS  8210.5434  345  23.7987 

ETA    =        2877  ETA   SQUARED   =  .0828 
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CRITERION   VARIABLE  5UM37 
BROKEN   DOWN  Bv        01  I  120 

-         ---         ------------        ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD   OEV      SUM  OF    SO  CASES 

0  296.0000  18.5000  8.3825      1054.0000  16 

1  8046. OOOO         24.3082  4.7298     7382,5680  331 

WITHIN  GROUPS   TOTAL  8342. OOOO         24.0403  4.9451     8436.5680  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D  F  SQUARE  F  SIG. 

BETWEEN  GROUPS  514.8672  1  514  8672         21.0547  OOOO 

WITHIN  GROUPS  8436.5680  345  24  4538 

ETA    =      .2398  ETA   SQUARED   =  .0575 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BV       01  I  12  1 

----------------------  ANALYSIS       OF  VARIANCE 

VALUE      LABEL                                                   SUM  MEAN          STD  OEV     SUM  OF   SQ  CASES 

0  1322. OOOO  20.9841           6.0335     2256.984  1  63 

1  7020. OOOO  24.7183           4.5951     5975  4648  284 

WITHIN  GROUPS   TOTAL                                  8342. OOOO  24.0403            4.8849     8232.4489  347 

•                                           ANALYSISOFVARIANCE  • 

SUM  OF  MEAN 

SOURCE                                               SQUARES                  D.F  SQUARE                     F  SIG. 

BETWEEN  GROUPS                            718.9862                    1  718.9862         30.1308  .OOOO 

WITHIN  GROUPS                             8232.4489                345  23  8622 

ETA   =      .2834         ETA  SQUARED   =  .0803 
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CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  By       01  I  122 

--------    ------  ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF    SO  CASES 

0  2563. OOGO         2  1.7203  5.6051     3675.77  12  118 

1  57  79  OOOO         25  2358  4   3495     4313.2664  229 

WITHIN  GROUPS   TOTAL  8342. OOOO  24.0403  4.8121      7989.0376  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  962  3976  1  962  3976         4 1   5603  OOOO 

WITHIN  GROUPS  7989  0376  345  23.1566 

ETA    =        3279  ETA   SQUARED   =  1075 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       Oil  123 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STO  DEV     SUM  OF   SO  CASES 

0  2963. OOOO         22.1119  5.5178     4049.3209  134 

1  5379. OOOO         25.2535  4.3925     4090.3099  213 

WITHIN  GROUPS  TOTAL  8342. OOOO         24.0403  4.8573     8139.6308  347 

ANALYSIS       OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  8  11.8044  1  8  11.8044  34.4085  OOOO 

WITHIN  GROUPS  8139.6308  345  23  5931 

ETA    =        3011  ETA   SQUARED    =  .0907 
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CRITERION    VARIABLE  5UM37 
BROKEN   DOWN   Bv  0III24 

---     ----------         ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD   DEV      SUM  OF    SO  CASES 

0  803  OOOO         21.7027  7.1253      1827,7297  37 

1  7539. OOOO         24.3194  4.7246     6897  3839  310 

WITHIN  GROUPS   TOTAL  8342  OOOO         24  0403  5.0289     8725.1136  347 

ANALYSIS        OF        VARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  OF  SQUARE  F  SIG. 

BETWEEN  GROUPS  226.3216  1  226.3216  8.9490  OO30 

WITHIN  GROUPS  8725.1136  345  25.2902 

ETA   =        1590         ETA  SQUARED   -  0253 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01  I  125 

--  ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF   SQ  CASES 

0  67. OOOO         11.1667  9.4745       448.8333  6 

1  8275  OOOO         24.2669  4.6938     7490.7155  34  1 

WITHIN  GROUPS  TOTAL  8342. OOOO         24.0403  4.7972     7939.5489  347 

*  ANALYSISOFVARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  1011.8863  1  1011   8863         43.9698  OOOO 

WITHIN  GROUPS  7939   5489  345  23  0132 

ETA   =      .3362         ETA  SQUARED   =  .1130 
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CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY  QIII2S 

----------------------  ANALYSIS        OF  VARIANCE 

VALUE      LABEL                                                   SUM  MEAN          STD  DEV     SUM  CF    SO  CASES 

0  1438. OOOO  19.6986           5.2670     1997  3699  73 

1  6904. OOOO  25    1971            4.3691      5211.3577  274 

WITHIN  GROUPS   TOTAL                                 8342  OOOO  24  0403            4.5711      7208.7275  347 

ANALYSIS        OF  VARIANCE  * 

SUM   OF  MEAN 

SOURCE                                               SQUARES                  D  F                SQUARE                     F  SIG. 

BETWEEN  GROUPS                            1742   7076  1             1742   7076          83.4036  .0000 

WITHIN   GROUPS                               7i08    7275                 345  20  8949 

ETA    =        4412  ETA   SQUARED    =  1947 


CRITERION  VAfilABLE  SUM37 
BROKEN  DOWN  BY        01  I  127 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF  SO  CASES 

0  870. OOOO         19.3333  6.0566     1614. OOOO  45 

1  7472. OOOO  24.7417  4.5355     6191.8543  302 

WITHIN  GROUPS   TOTAL  8342. OOOO  24.0403  4   7566     7805.8543  347 

*  ANALVSISOFVARIANCE  • 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F  SQUARE  F  SIG. 

BETWEEN  GROUPS  1145.5809  1  1145.5809         50.6319  .0000 

WITHIN  GROUPS  7805.8543  345  22.6257 

ETA   =       3577         ETA  SQUARED   '  1280 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY        QI  I  128 

----------------------        ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  2  107  OOOO         20.6569  5.3594     2900 . 9902  102 

1  6235. OOOO         25.4490  4.2449     4396.6122  245 

WITHIN   GROUPS    TOTAL  8342. OOOO  24   0403  4.5992      7297.6024  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F  SQUARE  F  SIG. 

BETWEEN  GROUPS  1653.8327  1  1653.8327         78.1863  .OOOO 

WITHIN  GROUPS  7297.6024  345  21  1525 

ETA   =       4298         ETA  SQUARED   =  .1848 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01 1  I  29 

---- --------   ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF   SO  CASES 

0  4883. OOOO         23.0330  5.2143     5736.7689  212 

1  3459. OOOO         25.6222  4.4569     2661.7333  135 

WITHIN  GROUPS  TOTAL  8342. OOOO         24.0403  4.9339     8398.5022  347 

ANALYSIS       OF        VARIAN  C  E 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  552.9330  1  552.9330         22.7138  OOOO 

WITHIN  GROUPS  8398.5022  345  24.3435 

ETA   «      .2485         ETA  SQUARED   '  0618 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY        01  I  130 

--------    -    -------------  ANALYSIS        OF  VARIANCE 

VALUE      LABEL                                                   SUM  MEAN          STD  DEV     SUM  OF   SO  CASES 

0  568.0OOO  20.2857           7.1277      1371.7143  28 

1  7774. GOOD  24.3699            4   7419     7150.3511  319 

WITHIN  GROUPS   TOTAL                                 8342. OOOO  24.0403            4.9701      8522.0654  347 

ANALYSIS         OF  VARIANCE 

SUM  OF  MEAN 

SOURCE                                            SQUARES                 D  F  SOUARE                   F  SIG. 

BETWEEN  GROUPS                             429.3698                     1  429.3698          17   3822  0000 

WITHIN  GROUPS                             8522.0654                345  24.7016 

ETA    =        2  190  ETA    SQUARED    =  .0480 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01  I  131 

----------------------        ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SQ  CASES 

0  4619. OOOO         22.3140  4.8814     4908.5894  207 

1  3723. OOOO         26.5929  4.2526     2513.7929  140 

WITHIN  GROUPS   TOTAL  8342. OOOO  24.0403  4.6383     7422.3822  347 

ANALYSIS       OF        VARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SOUARE  F  SIG. 

BETWEEN  GROUPS  1529.0529  1  1529.0529         71.0720  .0000 

WITHIN  GROUPS  7422.3822  345  21.5142 

ETA   =      .4133         ETA  SQUARED   =  1708 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY       Oil  132 

ANALYSIS       OF  VARIANCE 
VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF    SO  CASES 

0  1878  OOOO         20.6374  5.2968     2525.0330  91 

1  6464  OOOO         25  2500  4.4272     4998,0000  256 

WITHIN  GROUPS    TOTAL  8342  OOOO  24  0403  4.6697      7523.0330  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  1428.4022  1  1428  4022         65.5053  OOOO 

WITHIN  GROUPS  7523.0330  345  2  1.8059 

FTA   ^       3995         ETA  SQUARED   =  1596 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01 II 33 

ANALYSIS       OF  VARIANCE 
VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF   SO  CASES 

0  2853.0000         20.8248  4.9317     3307.7956  137 

1  5489.0000         26.1381  3.9754     3302.9952  2  10 

WITHIN  GROUPS   TOTAL  8342.0000  24.0403  4.3774     6610.7909  347 

ANALYSIS       OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  2340.6443  1  2340.6443        122.1521  OOOO 

WITHIN  GROUPS  6610.7909  345  19.1617 

ETA    -        5114  ETA   SQUARED    »  2615 
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CRITtRION   VARIABLE  5UM37 
BROKEN  DOWN  BY       01  I  I  34 

---    --   ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  49  10.0OOO  22.117  1  4.7608     5008 . 9550  222 

1  3432  OOOO         27  4560  3.6622      1663. 0080  125 

WITHIN  GROUPS  TOTAL  8342. OOOO         24.0403  4 . 397G     6671.9630  347 

ANALYSIS        OF        VARIANCE  • 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  2279  4722  1  2279  4722        117.8690  OOOO 

WITHIN  GROUPS  5571.9630  345  19  3390 

ETA    =        5046  ETA    SQUARED    =  2546 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01 II 35 

 -        ANALYSIS       OF  VARIANCE 

VALUE     LABEL  SUM  MEAN         STD  DEV     SUM  OF   SO  CASES 

0  307. OOOO         18  0588  7.4538       888.9412  17 

1  8035.0000         24.3485  4.7500     7422.9242  330 

WITHIN  GROUPS   TOTAL  8342. OOOO  24.0403  4.9084      83  11.8654  3^-7 

ANALYSIS        OF        VARIANCE  * 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F  SQUARE  F  SIG. 

BETWEEN  GROUPS  639.5697  1  639.5697         26.5466  .OOOO 

WITHIN  GROUPS  8311.8654  345  24.0924 

ETA    =      .2673  ETA   SQUARED    =  0714 
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CRITERION   VARIABLE  SUM37 
BROKEN  DOWN  BY  0III36 

----------  ------        ANALYSIS        OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  5TD  DEV     SUM  OF   SO  CASES 

0  973.0000  19.8571  5.6199      1516. OOOO  49 

1  7369. OOOO         24.7282  4.6555     6436.9832  298 

WITHIN  GROUPS   TOTAL  8342.0000         24.0403  4.8013     7952.9832  347 

ANALYSIS        OF  VARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  D.F.  SQUARE  F  SIG. 

BETWEEN  GROUPS  998.4519  1  998.4519         43.3128  OOOO 

WITHIN  GROUPS  7952.9832  345  23. 0521 

ETA    =        3340  ETA   SQUARED   =  1115 


CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  BY       01  I  I  37 

-    --    ------------   ANALYSIS       OF  VARIANCE 

VALUE      LABEL  SUM  MEAN  STD  DEV     SUM  OF   SO  CASES 

0  974. OOOO         20  2917  5.5350     1439.9167  48 

1  7368. OOOO         24.642  1  4.7518     6728.7090  299 

WITHIN  GROUPS   TOTAL  8342. OOOO         24  0403  4.8659     8168.6257  347 

ANALYSISOFVARIANCE 

SUM  OF  MEAN 
SOURCE  SQUARES  OF.  SQUARE  F  SIG. 

BETWEEN  GROUPS  782.8095  1  782.8095         33.0618  OOOO 

WITHIN  GROUPS  8168.6257  345  23.6772 

ETA   =      .2957         ETA  SQUARED   =  .0875 
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CRITERION  VARIABLE  SUM37 
BROKEN  DOWN  Bv  019 

----------------------  ANALVSIS       OF  VARIANCE 

VALUE      LABEL                                                   SUM  MEAN          STD  OEV     SUM  OF   SO  CASES 

1  7323  OOOO  24.2483           5.1812     8080.3742  302 

2  857. OOOO  22.5526           4.5184       755.3947  38 

3  14  1. OOOO  23.5000           1.2247           7.5000  6 

WITHIN  GROUPS   TOTAL                                 8321.0000  24.0491            5.0776     8843.2689  346 

ANALYSIS        OF        VARIANCE  * 

SUM  OF  MEAN 

SOURCE                                              SQUARES                  OF  SQUARE                     F  SIG 

BETWEEN  GROUPS                               98   3958                     2  49.4479            1.9179  1485 

WITHIN  GROUPS                           8843.2689               343  25.7821 

ETA   '      .1052         ETA  SQUARED   =  .0111 
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MULTIPLE        CLASSIFICATION        ANALYSIS  ••• 

SUM37 
BY        01  1 
01  I  1 


GRAND  ME  AN   »  24  19 


VARIABLE  ♦  CATEGORY 
01  1 


70 
65 
207 


UNADJUSTED 


ADJUSTED  FOR 
ADJUSTED  FOR  INDEPENDENTS 
INDEPENDENTS       +  COVARIATES 


DEV'N     ETA       DEV'N  BETA 


-0.80  -0  86 
1.58  1  58 

-0.22  -0.20 

0.17  0.17 


MULTIPLE  R  SQUARED 
MULTIPLE  R 


103 
62 
177 


0.18  0.22 

-0.16  -0.25 

-0  05  -0.04 

0.03  0.04 

0.030 
O.  173 


ANALYSIS       OF  VARIANCE 

SUM37 
BY        01  1 
01  I  1 


SOURCE   OF  VARIATION 

MAIN  EFFECTS 
01  1 
01  I  1 

2-WtY  INTERACTIONS 
011  0II1 

EXPLAINED 

RESIDUAL 

TOTAL 


SUM  OF 
SQUARES 

226  892 
221 .631 
9.  134 

33 . 701 
33 . 701 

260. 593 

7338 .053 

7598  646 


333 
34  1 


WEAN 
SQUARE 

56  723 

I  10.816 
4  .  567 

8  .  425 
8  ,  425 

32 . 574 

22 .036 

22  283 


SIGNIF 
F  OF  F 

2.574  0.038 
5.029  0  007 
0.207     0.8  13 


0.  382 
O.  382 


0  821 
0.  82  1 


1  . 478     0  164 


347  CASES  WERE  PROCESSED. 

5  CASES   (      1.4  PCT)   WERE  MISSING. 


181 


Pearson  Correlation  Coefficients 


01  1.1       01  1.2     01  1.3         Total  #  of 

Teachers 
Evaluated 


SUM37 

.2386 

.0279 

.0959 

.1063 

(71) 

(65) 

(211) 

(347  ) 

P  = 

=  .023 

p=.413 

p=.083 

p=.024 

PROCESS 

.2213 

a053 

.0722 

.1648 

(71) 

(65) 

(211) 

(347  ) 

P  = 

=  .032 

p=  .202 

p=.148 

p=.001 

IMPRVT 

.2505 

.0465 

.1015 

.0987 

(71) 

(65) 

(211) 

(  347  ) 

P  = 

=  .018 

p=.356 

p=.071 

p=.033 

TEACH 

.2380 

.0838 

.1237 

.0465 

(71) 

(65) 

(211) 

(347  ) 

P  = 

=  .023 

p=  .253 

p=.036 

p=.192 
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Pearson  Correlation  Coefficients 


QI  3 

014 

Qlb 

Q 1 1 U 

SUM37 

-.0067 

(  347  ) 
p-  .  4  dU 

.1445 

(  347  ) 
p-  .  UU4 

.0311 
(  347  ) 
p-  .282 

.0773 
(  347  ) 
p-  .  U  /  b 

-.0576 
(  345  ) 
p-  .14  3 

PROCESS 

.0147 
(347  ) 
p=.392 

.1518 
(347  ) 
p=.002 

.0887 
(347  ) 
p=.049 

.1507 
(  347  ) 
p=.002 

-.0301 
(  345) 
p=.289 

IMPRVT 

-  .0128 
(  347  ) 
p-  .  4  U  b 

.0883 
(  347  ) 

p-  .  U  dU 

.0554 

(  347  ) 

r^—  ICO 

p-  .  1  D  2 

.0644 
(  347  ) 

—  lie 

p-  .lib 

-  .0690 

(  345) 

_  _  inn 
p-  ,  1 U U 

TEACH 

-.0031 
(  347  ) 
p—  .  4  /  / 

.0273 

(347  ) 
p-  .  J  U  o 

.0508 

(  347  ) 

r^—  ITT 

p-  .  1  /  O 

.0285 
(  347  ) 
p-  .  z  y  y 

-.0651 

(345) 

p—  .14  4 

U  i  i  i 

r\  T  T  A 
0  i  i  4 

r\  T  T  C 
U  i  i  D 

U  i  1  b 

T  T  "7 
U  1  i  / 

SUM37 

-.2149 
(  347  ) 
p=.000 

.1286 

(  327) 
p=.010 

.2975 

(  338  ) 
p=  .000 

-.0930 
(  308) 
p=.052 

-  .1092 
(  340  ) 
p=.022 

PROCESS 

-  .2348 

(  347  ) 
p=  .000 

.1081 
(  327  ) 
p=  .025 

.2950 
(  338  ) 
p=  .000 

-.0520 

(  308  ) 
p=.181 

.1601 

(  340) 
p=  .002 

IMPRVT 

-.1881 

(  347  ) 
p=  .000 

.1512 

(  327  ) 
p=.003 

.2853 

(  338) 
p=  .000 

-.0462 
(  308  ) 
p=.210 

.0825 

(  340  ) 
p=.064 

TEACH 

-.1145 

(  347  ) 
p=.017 

.1873 

(327  ) 
p=.000 

.2725 

(338  ) 
p=.000 

-.0302 

(  308  ) 
p=.299 

.0868 

(  340  ) 
p=.055 

N.L  C.  -  B.N.C. 


^  3286  05334696  7 


